[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:05:20 dist-test-slave-bdps krb5kdc[8012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:05:20 dist-test-slave-bdps krb5kdc[8012](info): set up 2 sockets
Sep 29 03:05:20 dist-test-slave-bdps krb5kdc[8012](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 03:05:21.484457 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.955s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:05:21 dist-test-slave-bdps krb5kdc[8012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:05:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:05:21.615002 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:34695
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:34695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:21.739615 8025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:21.739899 8025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:21.739953 8025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:21.743955 8025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:21.744026 8025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:21.744053 8025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:21.744072 8025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:21.744093 8025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:21.749593 8025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:34695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:34695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:21.750895 8025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:21.752146 8025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:05:21.762336 8025 server_base.cc:1034] running on GCE node
W20260929 03:05:21.763109 8031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:21.763207 8033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:21.763254 8030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:21.763938 8025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:21.765326 8025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:21.766489 8025 hybrid_clock.cc:648] HybridClock initialized: now 1790651121766353 us; error 188 us; skew 500 ppm
Sep 29 03:05:21 dist-test-slave-bdps krb5kdc[8012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651121, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:21.769572 8025 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:05:21.770689 8025 webserver.cc:460] Webserver started at http://127.7.209.126:43107/ using document root <none> and password file <none>
I20260929 03:05:21.771349 8025 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:21.771420 8025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:21.771658 8025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:21.774168 8025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "832ef01326964446ab8c84f8eeee114a"
format_stamp: "Formatted at 2026-09-29 03:05:21 on dist-test-slave-bdps"
server_key: "247eac7dcb836c765c084fb6b169fefc"
server_key_iv: "aaf5c5b007ee207b23ed87eca9cacc0d"
server_key_version: "encryptionkey@0"
I20260929 03:05:21.774923 8025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "832ef01326964446ab8c84f8eeee114a"
format_stamp: "Formatted at 2026-09-29 03:05:21 on dist-test-slave-bdps"
server_key: "247eac7dcb836c765c084fb6b169fefc"
server_key_iv: "aaf5c5b007ee207b23ed87eca9cacc0d"
server_key_version: "encryptionkey@0"
I20260929 03:05:21.779193 8025 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:21.783488 8040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:21.785166 8025 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:05:21.785336 8025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "832ef01326964446ab8c84f8eeee114a"
format_stamp: "Formatted at 2026-09-29 03:05:21 on dist-test-slave-bdps"
server_key: "247eac7dcb836c765c084fb6b169fefc"
server_key_iv: "aaf5c5b007ee207b23ed87eca9cacc0d"
server_key_version: "encryptionkey@0"
I20260929 03:05:21.785486 8025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:21.815308 8025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:21.819154 8025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:21.819391 8025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:21.828351 8025 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:34695
I20260929 03:05:21.828393 8092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:34695 every 8 connection(s)
I20260929 03:05:21.829698 8025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:21.832671 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 8025
I20260929 03:05:21.832880 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:21.833984 8093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:21.834985 8005 external_mini_cluster.cc:1428] Setting key 0e548657e1a9465c7622659c9b43d4d6
I20260929 03:05:21.842728 8093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a: Bootstrap starting.
I20260929 03:05:21.846422 8093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:21.847398 8093 log.cc:824] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:21.850214 8093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a: No bootstrap required, opened a new log
I20260929 03:05:21.855146 8093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832ef01326964446ab8c84f8eeee114a" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34695 } }
I20260929 03:05:21.855422 8093 raft_consensus.cc:374] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:21.855540 8093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 832ef01326964446ab8c84f8eeee114a, State: Initialized, Role: FOLLOWER
I20260929 03:05:21.856148 8093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832ef01326964446ab8c84f8eeee114a" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34695 } }
I20260929 03:05:21.856328 8093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:21.856401 8093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:21.856511 8093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [term 0 FOLLOWER]: Advancing to term 1
Sep 29 03:05:21 dist-test-slave-bdps krb5kdc[8012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:21.860154 8093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832ef01326964446ab8c84f8eeee114a" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34695 } }
I20260929 03:05:21.860596 8093 leader_election.cc:302] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 832ef01326964446ab8c84f8eeee114a; no voters:
I20260929 03:05:21.860980 8093 leader_election.cc:288] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:21.861083 8098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:21.861739 8098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [term 1 LEADER]: Becoming Leader. State: Replica: 832ef01326964446ab8c84f8eeee114a, State: Running, Role: LEADER
I20260929 03:05:21.862888 8098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832ef01326964446ab8c84f8eeee114a" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34695 } }
I20260929 03:05:21.862499 8093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:21.864814 8100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "832ef01326964446ab8c84f8eeee114a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832ef01326964446ab8c84f8eeee114a" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34695 } } }
I20260929 03:05:21.864971 8100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:21.865348 8099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 832ef01326964446ab8c84f8eeee114a. Latest consensus state: current_term: 1 leader_uuid: "832ef01326964446ab8c84f8eeee114a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "832ef01326964446ab8c84f8eeee114a" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34695 } } }
I20260929 03:05:21.865458 8099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:21.865823 8109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:21.867901 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:21.840597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38418 (local address 127.7.209.126:34695)
0929 03:05:21.841263 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:21.841286 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:21.842530 (+ 1244us) server_negotiation.cc:407] Connection header received
0929 03:05:21.847416 (+ 4886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:21.847435 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:21.847871 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:21.848358 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:21.851754 (+ 3396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:21.852867 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:21.854758 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:21.855271 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:21.855396 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:21.860492 (+ 5096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:21.860499 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:21.860519 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:21.860558 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:21.862784 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:21.863474 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:21.863482 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:21.863488 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:21.863702 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:21.864125 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:21.864129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:21.864131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:21.864783 (+ 652us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:21.865020 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:21.866396 (+ 1376us) server_negotiation.cc:299] Negotiation successful
0929 03:05:21.866741 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":172,"threads_started":1}
I20260929 03:05:21.870054 8109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:21.877552 8109 catalog_manager.cc:1347] Generated new cluster ID: cc0b55236b844ebe83569056a881d3ca
I20260929 03:05:21.877660 8109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:21.897569 8109 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:21.899104 8109 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:21.907670 8109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 832ef01326964446ab8c84f8eeee114a: Generated new TSK 0
I20260929 03:05:21.908846 8109 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
W20260929 03:05:22.042150 8005 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.209.65: real 0.121s user 0.002s sys 0.005s
I20260929 03:05:22.042773 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34695
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:22.173816 8119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:22.174150 8119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:22.174199 8119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:22.178264 8119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:22.178323 8119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:22.178381 8119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:22.183843 8119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34695
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:22.185151 8119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:22.186483 8119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:22.198096 8125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:22.198349 8119 server_base.cc:1034] running on GCE node
W20260929 03:05:22.198096 8127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:22.198096 8124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:22.199000 8119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:22.199847 8119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:22.201021 8119 hybrid_clock.cc:648] HybridClock initialized: now 1790651122200993 us; error 43 us; skew 500 ppm
Sep 29 03:05:22 dist-test-slave-bdps krb5kdc[8012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651122, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:22.204351 8119 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:05:22.205533 8119 webserver.cc:460] Webserver started at http://127.7.209.65:32781/ using document root <none> and password file <none>
I20260929 03:05:22.206259 8119 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:22.206331 8119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:22.206573 8119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:22.209106 8119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "fedd2bc79644495e8e960925c441b1d4"
format_stamp: "Formatted at 2026-09-29 03:05:22 on dist-test-slave-bdps"
server_key: "33569db93b52994a65511dce460d484c"
server_key_iv: "692a16462ba8a3eb471a378c38a38eb1"
server_key_version: "encryptionkey@0"
I20260929 03:05:22.209782 8119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "fedd2bc79644495e8e960925c441b1d4"
format_stamp: "Formatted at 2026-09-29 03:05:22 on dist-test-slave-bdps"
server_key: "33569db93b52994a65511dce460d484c"
server_key_iv: "692a16462ba8a3eb471a378c38a38eb1"
server_key_version: "encryptionkey@0"
I20260929 03:05:22.214502 8119 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 03:05:22.217280 8134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:22.218693 8119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:22.218808 8119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "fedd2bc79644495e8e960925c441b1d4"
format_stamp: "Formatted at 2026-09-29 03:05:22 on dist-test-slave-bdps"
server_key: "33569db93b52994a65511dce460d484c"
server_key_iv: "692a16462ba8a3eb471a378c38a38eb1"
server_key_version: "encryptionkey@0"
I20260929 03:05:22.218906 8119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:22.234707 8119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:22.238389 8119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:22.238593 8119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:22.239370 8119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:22.240423 8119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:22.240473 8119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:22.240531 8119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:22.240557 8119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:22.256352 8119 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:46621
I20260929 03:05:22.256372 8247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:46621 every 8 connection(s)
I20260929 03:05:22.257416 8119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:22.261770 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 8119
I20260929 03:05:22.261873 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:05:22.262147 8005 external_mini_cluster.cc:1428] Setting key 197cb7931178b3604f7b37e46c276266
Sep 29 03:05:22 dist-test-slave-bdps krb5kdc[8012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651122, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:22.274391 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:22.259619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:56153 (local address 127.7.209.126:34695)
0929 03:05:22.259807 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:22.259813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:22.260863 (+ 1050us) server_negotiation.cc:407] Connection header received
0929 03:05:22.262459 (+ 1596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:22.262463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:22.262534 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:22.262617 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:22.264325 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.264985 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.266817 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.267208 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.267321 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:22.270458 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:22.270462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:22.270464 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:22.270491 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:22.271922 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:22.272590 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:22.272593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:22.272594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:22.272683 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:22.273061 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:22.273063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:22.273064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:22.273252 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:22.273323 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:22.274090 (+ 767us) server_negotiation.cc:299] Negotiation successful
0929 03:05:22.274240 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:05:22.275588 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:22.260111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:34695 (local address 127.7.209.65:56153)
0929 03:05:22.260696 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:22.260745 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:22.262198 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:22.262848 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:22.262859 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:22.263377 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:22.264166 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:22.264192 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.265143 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.265147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:22.266694 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:22.266704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.267356 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.267359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:22.267576 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:22.268283 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:22.268301 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:22.270262 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:22.272045 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:22.272051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:22.272073 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:22.272494 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:22.272766 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:22.272769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:22.272771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:22.272974 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:22.273354 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:22.273361 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:22.273795 (+ 434us) client_negotiation.cc:769] Sending connection context
0929 03:05:22.274062 (+ 267us) client_negotiation.cc:240] Negotiation successful
0929 03:05:22.274482 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":158,"threads_started":1}
I20260929 03:05:22.277007 8248 heartbeater.cc:347] Connected to a master server at 127.7.209.126:34695
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
I20260929 03:05:22.277297 8248 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:22.278002 8248 heartbeater.cc:511] Master 127.7.209.126:34695 requested a full tablet report, sending...
I20260929 03:05:22.279871 8057 ts_manager.cc:194] Registered new tserver with Master: fedd2bc79644495e8e960925c441b1d4 (127.7.209.65:46621)
I20260929 03:05:22.281502 8057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:56153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:05:22.294795 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34695
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:22.457280 8253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:22.457657 8253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:22.457732 8253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:22.461879 8253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:22.461943 8253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:22.462018 8253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:22.467968 8253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34695
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:22.469441 8253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:22.470795 8253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:22.482141 8261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:22.482291 8253 server_base.cc:1034] running on GCE node
W20260929 03:05:22.482483 8259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:22.482535 8258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:22.483021 8253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:22.483765 8253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:22.484918 8253 hybrid_clock.cc:648] HybridClock initialized: now 1790651122484893 us; error 41 us; skew 500 ppm
Sep 29 03:05:22 dist-test-slave-bdps krb5kdc[8012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651122, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:22.488306 8253 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:05:22.489434 8253 webserver.cc:460] Webserver started at http://127.7.209.66:40073/ using document root <none> and password file <none>
I20260929 03:05:22.490125 8253 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:22.490192 8253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:22.490424 8253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:22.493160 8253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "5e913040029b439f80b3687bf3b52a3f"
format_stamp: "Formatted at 2026-09-29 03:05:22 on dist-test-slave-bdps"
server_key: "8bf5355677dc6050adacde5a8aa940d7"
server_key_iv: "9106cc461a77c8020e374fac9da964bc"
server_key_version: "encryptionkey@0"
I20260929 03:05:22.493862 8253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "5e913040029b439f80b3687bf3b52a3f"
format_stamp: "Formatted at 2026-09-29 03:05:22 on dist-test-slave-bdps"
server_key: "8bf5355677dc6050adacde5a8aa940d7"
server_key_iv: "9106cc461a77c8020e374fac9da964bc"
server_key_version: "encryptionkey@0"
I20260929 03:05:22.498354 8253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260929 03:05:22.501060 8268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:22.502528 8253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:05:22.502633 8253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "5e913040029b439f80b3687bf3b52a3f"
format_stamp: "Formatted at 2026-09-29 03:05:22 on dist-test-slave-bdps"
server_key: "8bf5355677dc6050adacde5a8aa940d7"
server_key_iv: "9106cc461a77c8020e374fac9da964bc"
server_key_version: "encryptionkey@0"
I20260929 03:05:22.502743 8253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:22.532709 8253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:22.536228 8253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:22.536451 8253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:22.537245 8253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:22.538285 8253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:22.538335 8253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:22.538379 8253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:22.538408 8253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:22.555406 8253 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:34133
I20260929 03:05:22.555469 8381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:34133 every 8 connection(s)
I20260929 03:05:22.556564 8253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:22.562732 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 8253
I20260929 03:05:22.562852 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:05:22.563127 8005 external_mini_cluster.cc:1428] Setting key a1df1f7c5df64a7a8786f470a0836afd
Sep 29 03:05:22 dist-test-slave-bdps krb5kdc[8012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651122, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:22.573237 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:22.558872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:33111 (local address 127.7.209.126:34695)
0929 03:05:22.559055 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:22.559060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:22.560142 (+ 1082us) server_negotiation.cc:407] Connection header received
0929 03:05:22.561873 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:22.561877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:22.561942 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:22.562024 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:22.563696 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.564326 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.565859 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.566174 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.566275 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:22.569424 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:22.569427 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:22.569430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:22.569457 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:22.570983 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:22.571506 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:22.571509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:22.571510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:22.571606 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:22.571946 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:22.571949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:22.571951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:22.572193 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:22.572289 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:22.572893 (+ 604us) server_negotiation.cc:299] Negotiation successful
0929 03:05:22.573040 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 03:05:22.574358 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:22.559350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:34695 (local address 127.7.209.66:33111)
0929 03:05:22.559967 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:22.560017 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:22.561634 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:22.562246 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:22.562254 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:22.562779 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:22.563539 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:22.563559 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.564466 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.564469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:22.565725 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:22.565736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.566334 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.566337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:22.566534 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:22.567325 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:22.567350 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:22.569243 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:22.571098 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:22.571105 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:22.571124 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:22.571406 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:22.571706 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:22.571709 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:22.571712 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:22.571858 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:22.572300 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:22.572308 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:22.572622 (+ 314us) client_negotiation.cc:769] Sending connection context
0929 03:05:22.572859 (+ 237us) client_negotiation.cc:240] Negotiation successful
0929 03:05:22.573251 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":165,"threads_started":1}
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:05:22.575681 8382 heartbeater.cc:347] Connected to a master server at 127.7.209.126:34695
I20260929 03:05:22.575958 8382 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:22.576596 8382 heartbeater.cc:511] Master 127.7.209.126:34695 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
I20260929 03:05:22.577841 8057 ts_manager.cc:194] Registered new tserver with Master: 5e913040029b439f80b3687bf3b52a3f (127.7.209.66:34133)
I20260929 03:05:22.578687 8057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:33111
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:05:22.596239 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34695
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:43905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:22.759019 8387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:22.759375 8387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:22.759449 8387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:22.763679 8387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:22.763744 8387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:22.763826 8387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:22.769526 8387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:43905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34695
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:22.770805 8387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:22.772027 8387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:22.782629 8392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:22.782778 8393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:22.782663 8395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:22.782934 8387 server_base.cc:1034] running on GCE node
I20260929 03:05:22.783435 8387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:22.784296 8387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:22.785471 8387 hybrid_clock.cc:648] HybridClock initialized: now 1790651122785449 us; error 36 us; skew 500 ppm
Sep 29 03:05:22 dist-test-slave-bdps krb5kdc[8012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651122, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:22.788861 8387 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:05:22.790078 8387 webserver.cc:460] Webserver started at http://127.7.209.67:35299/ using document root <none> and password file <none>
I20260929 03:05:22.790779 8387 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:22.790856 8387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:22.791118 8387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:22.793735 8387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "4075cad771234bad94ab87c8dc0b0f30"
format_stamp: "Formatted at 2026-09-29 03:05:22 on dist-test-slave-bdps"
server_key: "7acf24393c230c8a580af6c76b477830"
server_key_iv: "c39682695ffd8e4cb3190dfbb44e736e"
server_key_version: "encryptionkey@0"
I20260929 03:05:22.794441 8387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "4075cad771234bad94ab87c8dc0b0f30"
format_stamp: "Formatted at 2026-09-29 03:05:22 on dist-test-slave-bdps"
server_key: "7acf24393c230c8a580af6c76b477830"
server_key_iv: "c39682695ffd8e4cb3190dfbb44e736e"
server_key_version: "encryptionkey@0"
I20260929 03:05:22.799124 8387 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:22.802332 8402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:22.803825 8387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:05:22.803944 8387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "4075cad771234bad94ab87c8dc0b0f30"
format_stamp: "Formatted at 2026-09-29 03:05:22 on dist-test-slave-bdps"
server_key: "7acf24393c230c8a580af6c76b477830"
server_key_iv: "c39682695ffd8e4cb3190dfbb44e736e"
server_key_version: "encryptionkey@0"
I20260929 03:05:22.804057 8387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:22.821514 8387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:22.825225 8387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:22.825461 8387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:22.826364 8387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:22.827440 8387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:22.827493 8387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:22.827571 8387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:22.827598 8387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:22.846463 8387 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:34853
I20260929 03:05:22.846557 8515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:34853 every 8 connection(s)
I20260929 03:05:22.847645 8387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:22.854944 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 8387
I20260929 03:05:22.855082 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:05:22.855402 8005 external_mini_cluster.cc:1428] Setting key 50e50e13160926a07220dced416d521a
Sep 29 03:05:22 dist-test-slave-bdps krb5kdc[8012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651122, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:22.863843 8095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:22.849872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:54009 (local address 127.7.209.126:34695)
0929 03:05:22.850020 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:22.850024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:22.851149 (+ 1125us) server_negotiation.cc:407] Connection header received
0929 03:05:22.852884 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:22.852888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:22.852964 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:22.853047 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:22.854783 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.855611 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.857200 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.857474 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.857559 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:22.860462 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:22.860465 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:22.860467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:22.860491 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:22.861949 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:22.862427 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:22.862431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:22.862432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:22.862511 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:22.862774 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:22.862776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:22.862777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:22.862950 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:22.863023 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:22.863586 (+ 563us) server_negotiation.cc:299] Negotiation successful
0929 03:05:22.863709 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 03:05:22.864933 8518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:22.850366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:34695 (local address 127.7.209.67:54009)
0929 03:05:22.850970 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:22.851019 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:22.852601 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:22.853292 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:22.853301 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:22.853861 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:22.854569 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:22.854596 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.855746 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.855749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:22.857080 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:22.857092 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.857666 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.857672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:22.857915 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:22.858652 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:22.858669 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:22.860321 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:22.862062 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:22.862067 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:22.862079 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:22.862329 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:22.862582 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:22.862584 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:22.862586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:22.862711 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:22.863057 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:22.863078 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:22.863375 (+ 297us) client_negotiation.cc:769] Sending connection context
0929 03:05:22.863552 (+ 177us) client_negotiation.cc:240] Negotiation successful
0929 03:05:22.863841 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":159,"threads_started":1}
I20260929 03:05:22.866206 8516 heartbeater.cc:347] Connected to a master server at 127.7.209.126:34695
I20260929 03:05:22.866467 8516 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:22.867050 8516 heartbeater.cc:511] Master 127.7.209.126:34695 requested a full tablet report, sending...
I20260929 03:05:22.868247 8057 ts_manager.cc:194] Registered new tserver with Master: 4075cad771234bad94ab87c8dc0b0f30 (127.7.209.67:34853)
I20260929 03:05:22.868891 8057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:54009
I20260929 03:05:22.870368 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 03:05:22 dist-test-slave-bdps krb5kdc[8012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651122, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 03:05:22 dist-test-slave-bdps krb5kdc[8012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651122, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
I20260929 03:05:22.899693 8523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:22.889959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34258 (local address 127.7.209.65:46621)
0929 03:05:22.890296 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:22.890304 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:22.890334 (+ 30us) server_negotiation.cc:407] Connection header received
0929 03:05:22.890399 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:22.890406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:22.890651 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:22.890800 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:22.891562 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.892360 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.893246 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.893718 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.893807 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:22.896280 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:22.896289 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:22.896303 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:22.896328 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:22.897928 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:22.898342 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:22.898347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:22.898349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:22.898435 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:22.898698 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:22.898702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:22.898704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:22.898956 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:22.899180 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:22.899352 (+ 172us) server_negotiation.cc:299] Negotiation successful
0929 03:05:22.899518 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":94,"threads_started":1}
W20260929 03:05:22.900832 8162 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34258
Sep 29 03:05:22 dist-test-slave-bdps krb5kdc[8012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 03:05:22 dist-test-slave-bdps krb5kdc[8012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
I20260929 03:05:22.931013 8523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:22.922103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.7.209.65:46621)
0929 03:05:22.922260 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:22.922265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:22.922426 (+ 161us) server_negotiation.cc:407] Connection header received
0929 03:05:22.922622 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:22.922625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:22.922681 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:22.922760 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:22.923631 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.924217 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.925102 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:22.925353 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:22.925448 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:22.928069 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:22.928073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:22.928075 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:22.928106 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:22.929423 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:22.929893 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:22.929897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:22.929898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:22.929983 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:22.930194 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:22.930196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:22.930197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:22.930372 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:22.930563 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:22.930747 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 03:05:22.930863 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:05:22.932235 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 8119
I20260929 03:05:22.939272 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 8253
I20260929 03:05:22.946511 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 8387
I20260929 03:05:22.951957 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 8025
2026-09-29T03:05:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2456 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:05:23 dist-test-slave-bdps krb5kdc[8530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:05:23 dist-test-slave-bdps krb5kdc[8530](info): set up 2 sockets
Sep 29 03:05:23 dist-test-slave-bdps krb5kdc[8530](info): commencing operation
krb5kdc: starting...
W20260929 03:05:23.921712 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:05:24 dist-test-slave-bdps krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:05:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:05:24.043649 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:46221
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:46221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:24.171474 8543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:24.171780 8543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:24.171824 8543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:24.175864 8543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:24.175925 8543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:24.175943 8543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:24.175959 8543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:24.175988 8543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:24.181360 8543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:46221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:46221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:24.182694 8543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:24.184015 8543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:24.194316 8548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:24.194382 8551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:24.194509 8543 server_base.cc:1034] running on GCE node
W20260929 03:05:24.194319 8549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:24.195295 8543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:24.196612 8543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:24.197767 8543 hybrid_clock.cc:648] HybridClock initialized: now 1790651124197739 us; error 42 us; skew 500 ppm
Sep 29 03:05:24 dist-test-slave-bdps krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:24.200922 8543 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:05:24.202071 8543 webserver.cc:460] Webserver started at http://127.7.209.126:38009/ using document root <none> and password file <none>
I20260929 03:05:24.202752 8543 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:24.202814 8543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:24.203029 8543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:24.205504 8543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "b361edc6420b41aaa2d382fc3d387e68"
format_stamp: "Formatted at 2026-09-29 03:05:24 on dist-test-slave-bdps"
server_key: "921a44a243b53d785efc06a3962fd1c6"
server_key_iv: "852cbbf1a0e297a5b7c1c6ece8961367"
server_key_version: "encryptionkey@0"
I20260929 03:05:24.206146 8543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "b361edc6420b41aaa2d382fc3d387e68"
format_stamp: "Formatted at 2026-09-29 03:05:24 on dist-test-slave-bdps"
server_key: "921a44a243b53d785efc06a3962fd1c6"
server_key_iv: "852cbbf1a0e297a5b7c1c6ece8961367"
server_key_version: "encryptionkey@0"
I20260929 03:05:24.210510 8543 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:24.213184 8558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:24.214681 8543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:24.214794 8543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "b361edc6420b41aaa2d382fc3d387e68"
format_stamp: "Formatted at 2026-09-29 03:05:24 on dist-test-slave-bdps"
server_key: "921a44a243b53d785efc06a3962fd1c6"
server_key_iv: "852cbbf1a0e297a5b7c1c6ece8961367"
server_key_version: "encryptionkey@0"
I20260929 03:05:24.214893 8543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:24.236092 8543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:24.239929 8543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:24.240114 8543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:24.247736 8543 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:46221
I20260929 03:05:24.247784 8610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:46221 every 8 connection(s)
I20260929 03:05:24.248845 8543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:24.250542 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 8543
I20260929 03:05:24.250646 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:24.250926 8005 external_mini_cluster.cc:1428] Setting key b8306e88699f175274d62c89bc05fbec
I20260929 03:05:24.252259 8611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:24.259466 8611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68: Bootstrap starting.
Sep 29 03:05:24 dist-test-slave-bdps krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:24.262496 8611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:24.263438 8611 log.cc:824] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:24.265965 8611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68: No bootstrap required, opened a new log
I20260929 03:05:24.266204 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:24.252428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50876 (local address 127.7.209.126:46221)
0929 03:05:24.252858 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:24.252875 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:24.252915 (+ 40us) server_negotiation.cc:407] Connection header received
0929 03:05:24.253671 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:24.253691 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:24.254059 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:24.254380 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:24.255288 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:24.256343 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:24.257481 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:24.258022 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:24.258154 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:24.260937 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:24.260944 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:24.260961 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:24.260993 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:24.262931 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:24.263394 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:24.263401 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:24.263407 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:24.263550 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:24.263842 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:24.263845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:24.263846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:24.264311 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:24.264506 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:24.264771 (+ 265us) server_negotiation.cc:299] Negotiation successful
0929 03:05:24.265094 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":136,"threads_started":1}
I20260929 03:05:24.270381 8611 raft_consensus.cc:348] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b361edc6420b41aaa2d382fc3d387e68" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46221 } }
I20260929 03:05:24.270645 8611 raft_consensus.cc:374] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:24.270691 8611 raft_consensus.cc:729] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b361edc6420b41aaa2d382fc3d387e68, State: Initialized, Role: FOLLOWER
I20260929 03:05:24.271134 8611 consensus_queue.cc:260] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b361edc6420b41aaa2d382fc3d387e68" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46221 } }
I20260929 03:05:24.271260 8611 raft_consensus.cc:388] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:24.271308 8611 raft_consensus.cc:482] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:24.271409 8611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:24.272496 8611 raft_consensus.cc:504] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b361edc6420b41aaa2d382fc3d387e68" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46221 } }
I20260929 03:05:24.272897 8611 leader_election.cc:302] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b361edc6420b41aaa2d382fc3d387e68; no voters:
I20260929 03:05:24.273320 8611 leader_election.cc:288] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:24.273423 8616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:24.273689 8616 raft_consensus.cc:686] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [term 1 LEADER]: Becoming Leader. State: Replica: b361edc6420b41aaa2d382fc3d387e68, State: Running, Role: LEADER
I20260929 03:05:24.274021 8616 consensus_queue.cc:237] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b361edc6420b41aaa2d382fc3d387e68" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46221 } }
I20260929 03:05:24.274732 8611 sys_catalog.cc:564] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:24.275874 8618 sys_catalog.cc:455] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b361edc6420b41aaa2d382fc3d387e68. Latest consensus state: current_term: 1 leader_uuid: "b361edc6420b41aaa2d382fc3d387e68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b361edc6420b41aaa2d382fc3d387e68" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46221 } } }
I20260929 03:05:24.276041 8618 sys_catalog.cc:458] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:24.275874 8617 sys_catalog.cc:455] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b361edc6420b41aaa2d382fc3d387e68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b361edc6420b41aaa2d382fc3d387e68" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46221 } } }
I20260929 03:05:24.276217 8617 sys_catalog.cc:458] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:24.276443 8625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:24.280321 8625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:24.287457 8625 catalog_manager.cc:1347] Generated new cluster ID: bd2926cc4e534756921f613a26dd0374
I20260929 03:05:24.287557 8625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:24.300804 8625 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:24.302016 8625 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:24.312314 8625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b361edc6420b41aaa2d382fc3d387e68: Generated new TSK 0
I20260929 03:05:24.313400 8625 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:05:24.346987 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:46221
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:24.475972 8637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:24.476343 8637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:24.476404 8637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:24.480623 8637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:24.480700 8637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:24.480773 8637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:24.486315 8637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:46221
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:24.487747 8637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:24.489060 8637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:24.499665 8643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:24.499707 8642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:24.499684 8645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:24.499990 8637 server_base.cc:1034] running on GCE node
I20260929 03:05:24.500414 8637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:24.501070 8637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:24.502220 8637 hybrid_clock.cc:648] HybridClock initialized: now 1790651124502203 us; error 32 us; skew 500 ppm
Sep 29 03:05:24 dist-test-slave-bdps krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:24.505223 8637 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:05:24.506369 8637 webserver.cc:460] Webserver started at http://127.7.209.65:41243/ using document root <none> and password file <none>
I20260929 03:05:24.507037 8637 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:24.507109 8637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:24.507373 8637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:24.509995 8637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "e24a225e88034470a16368cddf93814a"
format_stamp: "Formatted at 2026-09-29 03:05:24 on dist-test-slave-bdps"
server_key: "ac366f05f7810c5fd6db61078d0c2a9e"
server_key_iv: "304e7352b350f3c33f482611fb1cc066"
server_key_version: "encryptionkey@0"
I20260929 03:05:24.510659 8637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "e24a225e88034470a16368cddf93814a"
format_stamp: "Formatted at 2026-09-29 03:05:24 on dist-test-slave-bdps"
server_key: "ac366f05f7810c5fd6db61078d0c2a9e"
server_key_iv: "304e7352b350f3c33f482611fb1cc066"
server_key_version: "encryptionkey@0"
I20260929 03:05:24.515034 8637 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260929 03:05:24.517818 8652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:24.519333 8637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:05:24.519449 8637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "e24a225e88034470a16368cddf93814a"
format_stamp: "Formatted at 2026-09-29 03:05:24 on dist-test-slave-bdps"
server_key: "ac366f05f7810c5fd6db61078d0c2a9e"
server_key_iv: "304e7352b350f3c33f482611fb1cc066"
server_key_version: "encryptionkey@0"
I20260929 03:05:24.519554 8637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:24.533394 8637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:24.536358 8637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:24.536542 8637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:24.537190 8637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:24.538105 8637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:24.538162 8637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:24.538219 8637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:24.538245 8637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:24.548195 8637 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:38439
I20260929 03:05:24.548218 8765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:38439 every 8 connection(s)
I20260929 03:05:24.549345 8637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:24.554581 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 8637
I20260929 03:05:24.554700 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:05:24.554987 8005 external_mini_cluster.cc:1428] Setting key 861c452fddab2675fcf14b2da72600b4
Sep 29 03:05:24 dist-test-slave-bdps krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:24.564652 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:24.551600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:59707 (local address 127.7.209.126:46221)
0929 03:05:24.551758 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:24.551762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:24.552793 (+ 1031us) server_negotiation.cc:407] Connection header received
0929 03:05:24.554337 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:24.554341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:24.554406 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:24.554491 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:24.556015 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:24.556657 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:24.557905 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:24.558159 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:24.558251 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:24.561221 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:24.561225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:24.561227 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:24.561254 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:24.562640 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:24.563135 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:24.563138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:24.563139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:24.563217 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:24.563521 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:24.563525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:24.563526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:24.563713 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:24.563802 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:24.564327 (+ 525us) server_negotiation.cc:299] Negotiation successful
0929 03:05:24.564466 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 03:05:24.565590 8768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:24.552094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:46221 (local address 127.7.209.65:59707)
0929 03:05:24.552611 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:24.552669 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:24.554097 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:24.554715 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:24.554727 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:24.555195 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:24.555870 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:24.555888 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:24.556779 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:24.556782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:24.557790 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:24.557800 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:24.558319 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:24.558323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:24.558511 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:24.559269 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:24.559292 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:24.561060 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:24.562745 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:24.562751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:24.562770 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:24.563037 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:24.563297 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:24.563300 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:24.563302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:24.563437 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:24.563808 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:24.563815 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:24.564088 (+ 273us) client_negotiation.cc:769] Sending connection context
0929 03:05:24.564273 (+ 185us) client_negotiation.cc:240] Negotiation successful
0929 03:05:24.564634 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:05:24.566926 8766 heartbeater.cc:347] Connected to a master server at 127.7.209.126:46221
I20260929 03:05:24.567219 8766 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:24.567809 8766 heartbeater.cc:511] Master 127.7.209.126:46221 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
I20260929 03:05:24.569597 8575 ts_manager.cc:194] Registered new tserver with Master: e24a225e88034470a16368cddf93814a (127.7.209.65:38439)
I20260929 03:05:24.570909 8575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:59707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:05:24.584821 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:46221
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:24.709852 8771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:24.710247 8771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:24.710300 8771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:24.714636 8771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:24.714715 8771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:24.714800 8771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:24.720664 8771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:46221
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:24.722043 8771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:24.723338 8771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:24.733980 8779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:24.734001 8776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:24.734066 8771 server_base.cc:1034] running on GCE node
W20260929 03:05:24.733989 8777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:24.734673 8771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:24.735327 8771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:24.736480 8771 hybrid_clock.cc:648] HybridClock initialized: now 1790651124736462 us; error 32 us; skew 500 ppm
Sep 29 03:05:24 dist-test-slave-bdps krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:24.739300 8771 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:05:24.740298 8771 webserver.cc:460] Webserver started at http://127.7.209.66:36445/ using document root <none> and password file <none>
I20260929 03:05:24.740904 8771 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:24.740965 8771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:24.741180 8771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:24.743445 8771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "77ba1c669c904827bf25f44dea9af7c6"
format_stamp: "Formatted at 2026-09-29 03:05:24 on dist-test-slave-bdps"
server_key: "7b7a892400c4d1579be22259a7e55f5a"
server_key_iv: "668aa6a2e5a62df3d455f1e49aeb7d0a"
server_key_version: "encryptionkey@0"
I20260929 03:05:24.744019 8771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "77ba1c669c904827bf25f44dea9af7c6"
format_stamp: "Formatted at 2026-09-29 03:05:24 on dist-test-slave-bdps"
server_key: "7b7a892400c4d1579be22259a7e55f5a"
server_key_iv: "668aa6a2e5a62df3d455f1e49aeb7d0a"
server_key_version: "encryptionkey@0"
I20260929 03:05:24.748375 8771 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260929 03:05:24.751288 8786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:24.752691 8771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:05:24.752799 8771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "77ba1c669c904827bf25f44dea9af7c6"
format_stamp: "Formatted at 2026-09-29 03:05:24 on dist-test-slave-bdps"
server_key: "7b7a892400c4d1579be22259a7e55f5a"
server_key_iv: "668aa6a2e5a62df3d455f1e49aeb7d0a"
server_key_version: "encryptionkey@0"
I20260929 03:05:24.752908 8771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:24.771211 8771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:24.774761 8771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:24.774950 8771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:24.775686 8771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:24.776679 8771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:24.776736 8771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:24.776803 8771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:24.776832 8771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:24.786500 8771 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:42783
I20260929 03:05:24.786526 8899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:42783 every 8 connection(s)
I20260929 03:05:24.787647 8771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:24.792121 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 8771
I20260929 03:05:24.792238 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:05:24.792515 8005 external_mini_cluster.cc:1428] Setting key 5150a30e2aeefb7db1c808738dcf7570
Sep 29 03:05:24 dist-test-slave-bdps krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:05:24.803752 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:24.790074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:36015 (local address 127.7.209.126:46221)
0929 03:05:24.790248 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:24.790253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:24.791468 (+ 1215us) server_negotiation.cc:407] Connection header received
0929 03:05:24.793074 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:24.793078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:24.793144 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:24.793226 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:24.794744 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:24.795315 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:24.796897 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:24.797144 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:24.797227 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:24.800141 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:24.800145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:24.800147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:24.800176 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:24.801583 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:24.802120 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:24.802124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:24.802126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:24.802214 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:24.802529 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:24.802532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:24.802534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:24.802757 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:24.802844 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:24.803404 (+ 560us) server_negotiation.cc:299] Negotiation successful
0929 03:05:24.803550 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 03:05:24.804764 8902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:24.790619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:46221 (local address 127.7.209.66:36015)
0929 03:05:24.791287 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:24.791342 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:24.792821 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:24.793449 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:24.793458 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:24.793942 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:24.794589 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:24.794608 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:24.795474 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:24.795478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:24.796754 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:24.796766 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:24.797300 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:24.797304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:24.797511 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:24.798196 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:24.798214 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:24.799963 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:24.801726 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:24.801733 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:24.801753 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:24.802025 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:24.802296 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:24.802299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:24.802302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:24.802447 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:24.802853 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:24.802861 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:24.803154 (+ 293us) client_negotiation.cc:769] Sending connection context
0929 03:05:24.803345 (+ 191us) client_negotiation.cc:240] Negotiation successful
0929 03:05:24.803724 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":136,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
I20260929 03:05:24.806123 8900 heartbeater.cc:347] Connected to a master server at 127.7.209.126:46221
I20260929 03:05:24.806418 8900 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:24.806989 8900 heartbeater.cc:511] Master 127.7.209.126:46221 requested a full tablet report, sending...
I20260929 03:05:24.808264 8575 ts_manager.cc:194] Registered new tserver with Master: 77ba1c669c904827bf25f44dea9af7c6 (127.7.209.66:42783)
I20260929 03:05:24.808888 8575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:36015
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:05:24.821388 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:46221
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:24.952103 8905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:24.952426 8905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:24.952491 8905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:24.956609 8905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:24.956676 8905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:24.956755 8905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:24.962152 8905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:46221
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:24.963414 8905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:24.964663 8905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:24.975602 8913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:24.975652 8911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:24.975706 8905 server_base.cc:1034] running on GCE node
W20260929 03:05:24.975652 8910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:24.976359 8905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:24.977077 8905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:24.978250 8905 hybrid_clock.cc:648] HybridClock initialized: now 1790651124978226 us; error 36 us; skew 500 ppm
Sep 29 03:05:24 dist-test-slave-bdps krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:24.981120 8905 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:05:24.982134 8905 webserver.cc:460] Webserver started at http://127.7.209.67:43023/ using document root <none> and password file <none>
I20260929 03:05:24.982748 8905 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:24.982810 8905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:24.983062 8905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:24.985435 8905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "a6dc20957df54e12be9acdbf697fb20b"
format_stamp: "Formatted at 2026-09-29 03:05:24 on dist-test-slave-bdps"
server_key: "db5e80d87bcd53fff62d72d26b53dcfd"
server_key_iv: "6bacdbce1f3a2e236ae568cff7b40177"
server_key_version: "encryptionkey@0"
I20260929 03:05:24.986123 8905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "a6dc20957df54e12be9acdbf697fb20b"
format_stamp: "Formatted at 2026-09-29 03:05:24 on dist-test-slave-bdps"
server_key: "db5e80d87bcd53fff62d72d26b53dcfd"
server_key_iv: "6bacdbce1f3a2e236ae568cff7b40177"
server_key_version: "encryptionkey@0"
I20260929 03:05:24.991088 8905 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260929 03:05:24.994318 8920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:24.995863 8905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:05:24.995993 8905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "a6dc20957df54e12be9acdbf697fb20b"
format_stamp: "Formatted at 2026-09-29 03:05:24 on dist-test-slave-bdps"
server_key: "db5e80d87bcd53fff62d72d26b53dcfd"
server_key_iv: "6bacdbce1f3a2e236ae568cff7b40177"
server_key_version: "encryptionkey@0"
I20260929 03:05:24.996109 8905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:25.027140 8905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:25.030915 8905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:25.031107 8905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:25.031852 8905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:25.032877 8905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:25.032929 8905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:25.032979 8905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:25.033005 8905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:25.043087 8905 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:38425
I20260929 03:05:25.043112 9033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:38425 every 8 connection(s)
I20260929 03:05:25.044121 8905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:25.049078 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 8905
I20260929 03:05:25.049228 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:05:25.049573 8005 external_mini_cluster.cc:1428] Setting key f174aaf251e779d5dc0758f84179f6d7
Sep 29 03:05:25 dist-test-slave-bdps krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651124, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:25.059291 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.046198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:55193 (local address 127.7.209.126:46221)
0929 03:05:25.046336 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:25.046340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:25.047306 (+ 966us) server_negotiation.cc:407] Connection header received
0929 03:05:25.048703 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:25.048705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:25.048763 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:25.048842 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:25.050568 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.051308 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.052644 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.052880 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.052965 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.055478 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:25.055481 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:25.055483 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:25.055511 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:25.056918 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:25.057492 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:25.057496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:25.057498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:25.057629 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:25.058071 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:25.058075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:25.058076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:25.058284 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:25.058377 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:25.058964 (+ 587us) server_negotiation.cc:299] Negotiation successful
0929 03:05:25.059104 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:05:25.060308 9036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.046623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:46221 (local address 127.7.209.67:55193)
0929 03:05:25.047115 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:25.047173 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:25.048484 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:25.049083 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:25.049093 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:25.049648 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:25.050397 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:25.050416 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.051461 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.051465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:25.052539 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:25.052548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.052998 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.053000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:25.053157 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.053780 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:25.053797 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:25.055332 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:25.057019 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:25.057025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:25.057044 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:25.057358 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:25.057745 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:25.057749 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:25.057751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:25.057935 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:25.058387 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:25.058395 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:25.058690 (+ 295us) client_negotiation.cc:769] Sending connection context
0929 03:05:25.058910 (+ 220us) client_negotiation.cc:240] Negotiation successful
0929 03:05:25.059275 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":122,"threads_started":1}
I20260929 03:05:25.061738 9034 heartbeater.cc:347] Connected to a master server at 127.7.209.126:46221
I20260929 03:05:25.062016 9034 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:25.062690 9034 heartbeater.cc:511] Master 127.7.209.126:46221 requested a full tablet report, sending...
I20260929 03:05:25.063886 8575 ts_manager.cc:194] Registered new tserver with Master: a6dc20957df54e12be9acdbf697fb20b (127.7.209.67:38425)
I20260929 03:05:25.064546 8575 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:55193
I20260929 03:05:25.075690 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:05:25.087025 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.080044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50890 (local address 127.7.209.126:46221)
0929 03:05:25.080179 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:25.080183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:25.080350 (+ 167us) server_negotiation.cc:407] Connection header received
0929 03:05:25.080463 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:25.080465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:25.080513 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:25.080582 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:25.081289 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.081905 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.082745 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.082965 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.083037 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.084378 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:25.084381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:25.084383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:25.084407 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:25.085664 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:25.086024 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:25.086027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:25.086028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:25.086099 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:25.086275 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:25.086277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:25.086278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:25.086511 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:25.086614 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:25.086782 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 03:05:25.086881 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:05:25.092397 8575 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:05:25.095956 8575 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:25.112224 9050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.106436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50966 (local address 127.7.209.67:38425)
0929 03:05:25.107046 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:25.107061 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:25.107126 (+ 65us) server_negotiation.cc:407] Connection header received
0929 03:05:25.107393 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:25.107400 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:25.107692 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:25.108213 (+ 521us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:25.108399 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.109223 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.110658 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.111456 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.111566 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.111751 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:25.111911 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 03:05:25.112032 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":123,"threads_started":1}
I20260929 03:05:25.112223 9049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.105744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:38439 (local address 127.0.0.1:54666)
0929 03:05:25.106978 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:25.107040 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:25.107218 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:25.107842 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:25.107849 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:25.107871 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:25.108076 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:25.108083 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.109653 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.109657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:25.110922 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:25.110932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.111801 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.111804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:25.111898 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.111919 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 03:05:25.111959 (+ 40us) client_negotiation.cc:240] Negotiation successful
0929 03:05:25.112013 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":8,"spinlock_wait_cycles":5632,"thread_start_us":96,"threads_started":1}
I20260929 03:05:25.112491 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.105926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:42783 (local address 127.0.0.1:38464)
0929 03:05:25.107451 (+ 1525us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:25.107463 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:25.107561 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:25.108198 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:25.108201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:25.108218 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:25.108297 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:25.108302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.109761 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.109764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:25.110922 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:25.110932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.112099 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.112103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:25.112193 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.112213 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 03:05:25.112296 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 03:05:25.112371 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1470,"mutex_wait_us":544,"thread_start_us":86,"threads_started":1}
I20260929 03:05:25.112609 9048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.106210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38464 (local address 127.7.209.66:42783)
0929 03:05:25.107528 (+ 1318us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:25.107553 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:25.107577 (+ 24us) server_negotiation.cc:407] Connection header received
0929 03:05:25.107656 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:25.107660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:25.107842 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:25.107991 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:25.108415 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.109588 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.111050 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.111867 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.111968 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.112120 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:25.112333 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 03:05:25.112451 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1192,"thread_start_us":116,"threads_started":1}
I20260929 03:05:25.112761 9045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.105926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:38425 (local address 127.0.0.1:50966)
0929 03:05:25.106977 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:25.107040 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:25.107219 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:25.107861 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:25.107865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:25.107881 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:25.108077 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:25.108085 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.109364 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.109367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:25.110525 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:25.110534 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.111586 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.111590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:25.111700 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.111753 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 03:05:25.111870 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 03:05:25.112013 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"mutex_wait_us":29,"spinlock_wait_cycles":6272,"thread_start_us":135,"threads_started":1}
I20260929 03:05:25.113291 9047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.105977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54666 (local address 127.7.209.65:38439)
0929 03:05:25.106823 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:25.106833 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:25.107150 (+ 317us) server_negotiation.cc:407] Connection header received
0929 03:05:25.107394 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:25.107401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:25.107694 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:25.107858 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:25.108197 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.109478 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.111078 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.111671 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.112659 (+ 988us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.112823 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:25.112875 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 03:05:25.113124 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":167,"threads_started":1}
I20260929 03:05:25.116062 8834 tablet_service.cc:1467] Processing CreateTablet for tablet 41afcf50fff64066a6b6937c832e8b6b (DEFAULT_TABLE table=test-table [id=d0649e0ea893452b81d859a647f2b505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:25.116062 8968 tablet_service.cc:1467] Processing CreateTablet for tablet 41afcf50fff64066a6b6937c832e8b6b (DEFAULT_TABLE table=test-table [id=d0649e0ea893452b81d859a647f2b505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:25.117285 8834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41afcf50fff64066a6b6937c832e8b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:25.117288 8968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41afcf50fff64066a6b6937c832e8b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:25.118643 8700 tablet_service.cc:1467] Processing CreateTablet for tablet 41afcf50fff64066a6b6937c832e8b6b (DEFAULT_TABLE table=test-table [id=d0649e0ea893452b81d859a647f2b505]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:25.119628 8700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41afcf50fff64066a6b6937c832e8b6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:25.124576 9051 tablet_bootstrap.cc:492] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b: Bootstrap starting.
I20260929 03:05:25.125963 9052 tablet_bootstrap.cc:492] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6: Bootstrap starting.
I20260929 03:05:25.126911 9053 tablet_bootstrap.cc:492] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a: Bootstrap starting.
I20260929 03:05:25.128203 9051 tablet_bootstrap.cc:654] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:25.128671 9052 tablet_bootstrap.cc:654] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:25.129190 9051 log.cc:824] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:25.129316 9053 tablet_bootstrap.cc:654] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:25.129415 9052 log.cc:824] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:25.130160 9053 log.cc:824] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:25.131415 9051 tablet_bootstrap.cc:492] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b: No bootstrap required, opened a new log
I20260929 03:05:25.131410 9052 tablet_bootstrap.cc:492] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6: No bootstrap required, opened a new log
I20260929 03:05:25.131619 9052 ts_tablet_manager.cc:1397] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:05:25.131628 9051 ts_tablet_manager.cc:1397] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260929 03:05:25.131963 9053 tablet_bootstrap.cc:492] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a: No bootstrap required, opened a new log
I20260929 03:05:25.132143 9053 ts_tablet_manager.cc:1397] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 03:05:25.135788 9053 raft_consensus.cc:348] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24a225e88034470a16368cddf93814a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38439 } } peers { permanent_uuid: "a6dc20957df54e12be9acdbf697fb20b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 38425 } } peers { permanent_uuid: "77ba1c669c904827bf25f44dea9af7c6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 42783 } }
I20260929 03:05:25.135794 9051 raft_consensus.cc:348] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24a225e88034470a16368cddf93814a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38439 } } peers { permanent_uuid: "a6dc20957df54e12be9acdbf697fb20b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 38425 } } peers { permanent_uuid: "77ba1c669c904827bf25f44dea9af7c6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 42783 } }
I20260929 03:05:25.135794 9052 raft_consensus.cc:348] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24a225e88034470a16368cddf93814a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38439 } } peers { permanent_uuid: "a6dc20957df54e12be9acdbf697fb20b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 38425 } } peers { permanent_uuid: "77ba1c669c904827bf25f44dea9af7c6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 42783 } }
I20260929 03:05:25.136041 9053 raft_consensus.cc:374] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:25.136065 9052 raft_consensus.cc:374] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:25.136062 9051 raft_consensus.cc:374] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:25.136106 9053 raft_consensus.cc:729] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e24a225e88034470a16368cddf93814a, State: Initialized, Role: FOLLOWER
I20260929 03:05:25.136121 9051 raft_consensus.cc:729] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6dc20957df54e12be9acdbf697fb20b, State: Initialized, Role: FOLLOWER
I20260929 03:05:25.136130 9052 raft_consensus.cc:729] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77ba1c669c904827bf25f44dea9af7c6, State: Initialized, Role: FOLLOWER
I20260929 03:05:25.136583 9053 consensus_queue.cc:260] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24a225e88034470a16368cddf93814a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38439 } } peers { permanent_uuid: "a6dc20957df54e12be9acdbf697fb20b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 38425 } } peers { permanent_uuid: "77ba1c669c904827bf25f44dea9af7c6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 42783 } }
I20260929 03:05:25.136585 9051 consensus_queue.cc:260] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24a225e88034470a16368cddf93814a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38439 } } peers { permanent_uuid: "a6dc20957df54e12be9acdbf697fb20b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 38425 } } peers { permanent_uuid: "77ba1c669c904827bf25f44dea9af7c6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 42783 } }
I20260929 03:05:25.136585 9052 consensus_queue.cc:260] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24a225e88034470a16368cddf93814a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38439 } } peers { permanent_uuid: "a6dc20957df54e12be9acdbf697fb20b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 38425 } } peers { permanent_uuid: "77ba1c669c904827bf25f44dea9af7c6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 42783 } }
I20260929 03:05:25.137308 9034 heartbeater.cc:503] Master 127.7.209.126:46221 was elected leader, sending a full tablet report...
I20260929 03:05:25.137589 8900 heartbeater.cc:503] Master 127.7.209.126:46221 was elected leader, sending a full tablet report...
I20260929 03:05:25.137709 9053 ts_tablet_manager.cc:1428] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260929 03:05:25.137710 9052 ts_tablet_manager.cc:1428] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 03:05:25.138019 8766 heartbeater.cc:503] Master 127.7.209.126:46221 was elected leader, sending a full tablet report...
I20260929 03:05:25.139000 9051 ts_tablet_manager.cc:1428] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260929 03:05:25.248142 9058 raft_consensus.cc:482] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:25.248427 9058 raft_consensus.cc:504] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24a225e88034470a16368cddf93814a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38439 } } peers { permanent_uuid: "a6dc20957df54e12be9acdbf697fb20b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 38425 } } peers { permanent_uuid: "77ba1c669c904827bf25f44dea9af7c6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 42783 } }
I20260929 03:05:25.249903 9058 leader_election.cc:288] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e24a225e88034470a16368cddf93814a (127.7.209.65:38439), 77ba1c669c904827bf25f44dea9af7c6 (127.7.209.66:42783)
I20260929 03:05:25.254580 9036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.250323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:42783 (local address 127.7.209.67:37579)
0929 03:05:25.250547 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:25.250566 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:25.250704 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:25.251027 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:25.251031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:25.251056 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:25.251217 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:25.251234 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.252196 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.252200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:25.253227 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:25.253237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.254084 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.254087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:25.254236 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.254261 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 03:05:25.254345 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 03:05:25.254432 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260929 03:05:25.254987 9048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.250435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:37579 (local address 127.7.209.66:42783)
0929 03:05:25.250576 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:25.250581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:25.250799 (+ 218us) server_negotiation.cc:407] Connection header received
0929 03:05:25.250864 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:25.250867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:25.250923 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:25.250991 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:25.251376 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.252093 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.253378 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.253970 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.254100 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.254171 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:25.254764 (+ 593us) server_negotiation.cc:299] Negotiation successful
0929 03:05:25.254860 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 03:05:25.255904 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.250323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:38439 (local address 127.7.209.67:53623)
0929 03:05:25.251072 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:25.251086 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:25.251243 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:25.251484 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:25.251487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:25.251505 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:25.251649 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:25.251656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.252739 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.252742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:25.253938 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:25.253948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.255432 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.255439 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:25.255569 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.255595 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 03:05:25.255668 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 03:05:25.255736 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":284,"thread_start_us":168,"threads_started":1}
I20260929 03:05:25.255777 8854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41afcf50fff64066a6b6937c832e8b6b" candidate_uuid: "a6dc20957df54e12be9acdbf697fb20b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77ba1c669c904827bf25f44dea9af7c6" is_pre_election: true
I20260929 03:05:25.255932 9047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.250548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:53623 (local address 127.7.209.65:38439)
0929 03:05:25.250809 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:25.250814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:25.251245 (+ 431us) server_negotiation.cc:407] Connection header received
0929 03:05:25.251326 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:25.251333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:25.251390 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:25.251463 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:25.251825 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.252614 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.254052 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.254801 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.254908 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.254955 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:25.255716 (+ 761us) server_negotiation.cc:299] Negotiation successful
0929 03:05:25.255801 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260929 03:05:25.256199 8854 raft_consensus.cc:2454] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6dc20957df54e12be9acdbf697fb20b in term 0.
I20260929 03:05:25.256796 8922 leader_election.cc:302] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77ba1c669c904827bf25f44dea9af7c6, a6dc20957df54e12be9acdbf697fb20b; no voters:
I20260929 03:05:25.256954 8720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41afcf50fff64066a6b6937c832e8b6b" candidate_uuid: "a6dc20957df54e12be9acdbf697fb20b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e24a225e88034470a16368cddf93814a" is_pre_election: true
I20260929 03:05:25.257117 9058 raft_consensus.cc:2781] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:25.257208 9058 raft_consensus.cc:482] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:25.257265 9058 raft_consensus.cc:3037] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:25.257294 8720 raft_consensus.cc:2454] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6dc20957df54e12be9acdbf697fb20b in term 0.
I20260929 03:05:25.258910 9058 raft_consensus.cc:504] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24a225e88034470a16368cddf93814a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38439 } } peers { permanent_uuid: "a6dc20957df54e12be9acdbf697fb20b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 38425 } } peers { permanent_uuid: "77ba1c669c904827bf25f44dea9af7c6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 42783 } }
I20260929 03:05:25.259232 9058 leader_election.cc:288] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [CANDIDATE]: Term 1 election: Requested vote from peers e24a225e88034470a16368cddf93814a (127.7.209.65:38439), 77ba1c669c904827bf25f44dea9af7c6 (127.7.209.66:42783)
I20260929 03:05:25.259536 8720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41afcf50fff64066a6b6937c832e8b6b" candidate_uuid: "a6dc20957df54e12be9acdbf697fb20b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e24a225e88034470a16368cddf93814a"
I20260929 03:05:25.259578 8854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41afcf50fff64066a6b6937c832e8b6b" candidate_uuid: "a6dc20957df54e12be9acdbf697fb20b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77ba1c669c904827bf25f44dea9af7c6"
I20260929 03:05:25.259658 8720 raft_consensus.cc:3037] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:25.259686 8854 raft_consensus.cc:3037] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:25.261143 8720 raft_consensus.cc:2454] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6dc20957df54e12be9acdbf697fb20b in term 1.
I20260929 03:05:25.261425 8924 leader_election.cc:302] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6dc20957df54e12be9acdbf697fb20b, e24a225e88034470a16368cddf93814a; no voters:
I20260929 03:05:25.261464 8854 raft_consensus.cc:2454] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6dc20957df54e12be9acdbf697fb20b in term 1.
I20260929 03:05:25.261621 9058 raft_consensus.cc:2781] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:25.261890 9058 raft_consensus.cc:686] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [term 1 LEADER]: Becoming Leader. State: Replica: a6dc20957df54e12be9acdbf697fb20b, State: Running, Role: LEADER
I20260929 03:05:25.262265 9058 consensus_queue.cc:237] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24a225e88034470a16368cddf93814a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38439 } } peers { permanent_uuid: "a6dc20957df54e12be9acdbf697fb20b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 38425 } } peers { permanent_uuid: "77ba1c669c904827bf25f44dea9af7c6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 42783 } }
I20260929 03:05:25.265641 8575 catalog_manager.cc:5495] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b reported cstate change: term changed from 0 to 1, leader changed from <none> to a6dc20957df54e12be9acdbf697fb20b (127.7.209.67). New cstate: current_term: 1 leader_uuid: "a6dc20957df54e12be9acdbf697fb20b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e24a225e88034470a16368cddf93814a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6dc20957df54e12be9acdbf697fb20b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 38425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77ba1c669c904827bf25f44dea9af7c6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 42783 } health_report { overall_health: UNKNOWN } } }
W20260929 03:05:25.289098 8901 tablet.cc:2369] T 41afcf50fff64066a6b6937c832e8b6b P 77ba1c669c904827bf25f44dea9af7c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:05:25.295200 9035 tablet.cc:2369] T 41afcf50fff64066a6b6937c832e8b6b P a6dc20957df54e12be9acdbf697fb20b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 03:05:25 dist-test-slave-bdps krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
W20260929 03:05:25.300921 8767 tablet.cc:2369] T 41afcf50fff64066a6b6937c832e8b6b P e24a225e88034470a16368cddf93814a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:05:25.304601 9047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.294126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54672 (local address 127.7.209.65:38439)
0929 03:05:25.294283 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:25.294288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:25.294563 (+ 275us) server_negotiation.cc:407] Connection header received
0929 03:05:25.294733 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:25.294737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:25.294794 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:25.294870 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:25.295801 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.296373 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.297409 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.297668 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.297761 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.300656 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:25.300672 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:25.300691 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:25.300720 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:25.302691 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:25.303130 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:25.303139 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:25.303141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:25.303295 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:25.303557 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:25.303560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:25.303562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:25.303871 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:25.304095 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:25.304287 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 03:05:25.304424 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 03:05:25 dist-test-slave-bdps krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651125, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 03:05:25 dist-test-slave-bdps krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651125, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
I20260929 03:05:25.335474 9047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.327560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54684 (local address 127.7.209.65:38439)
0929 03:05:25.327748 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:25.327753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:25.327872 (+ 119us) server_negotiation.cc:407] Connection header received
0929 03:05:25.328008 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:25.328011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:25.328061 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:25.328136 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:25.328825 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.329387 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.330261 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.330490 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.330569 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.332859 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:25.332862 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:25.332865 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:25.332890 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:25.334099 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:25.334448 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:25.334450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:25.334452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:25.334522 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:25.334701 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:25.334703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:25.334705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:25.334859 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:25.334958 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:25.335180 (+ 222us) server_negotiation.cc:299] Negotiation successful
0929 03:05:25.335285 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Sep 29 03:05:25 dist-test-slave-bdps krb5kdc[8530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 03:05:25 dist-test-slave-bdps krb5kdc[8530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
I20260929 03:05:25.363269 9047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:25.354897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.7.209.65:38439)
0929 03:05:25.355076 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:25.355082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:25.355217 (+ 135us) server_negotiation.cc:407] Connection header received
0929 03:05:25.355380 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:25.355383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:25.355439 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:25.355519 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:25.356262 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.356841 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.357786 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:25.358015 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:25.358102 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:25.360639 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:25.360643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:25.360645 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:25.360672 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:25.361906 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:25.362270 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:25.362272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:25.362274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:25.362358 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:25.362550 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:25.362552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:25.362553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:25.362737 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:25.362837 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:25.363011 (+ 174us) server_negotiation.cc:299] Negotiation successful
0929 03:05:25.363120 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260929 03:05:25.364976 8680 tablet_service.cc:2908] Rejecting scan request for tablet 41afcf50fff64066a6b6937c832e8b6b: Uninitialized: clean time has not yet been initialized
I20260929 03:05:25.367305 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 8637
I20260929 03:05:25.373689 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 8771
I20260929 03:05:25.379267 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 8905
I20260929 03:05:25.384032 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 8543
2026-09-29T03:05:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2432 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:05:25 dist-test-slave-bdps krb5kdc[9075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:05:25 dist-test-slave-bdps krb5kdc[9075](info): set up 2 sockets
Sep 29 03:05:25 dist-test-slave-bdps krb5kdc[9075](info): commencing operation
krb5kdc: starting...
W20260929 03:05:26.345558 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:05:26 dist-test-slave-bdps krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:05:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:05:26.468446 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:36121
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:36121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:26.595047 9088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:26.595436 9088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:26.595498 9088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:26.599746 9088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:26.599800 9088 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 03:05:26.599825 9088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:26.599848 9088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:26.599872 9088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:26.599900 9088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:26.605526 9088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:36121
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:36121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:26.606899 9088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:26.608204 9088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:26.617904 9096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:26.617975 9093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:26.618101 9088 server_base.cc:1034] running on GCE node
W20260929 03:05:26.617904 9094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:26.618866 9088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:26.620155 9088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:26.621325 9088 hybrid_clock.cc:648] HybridClock initialized: now 1790651126621307 us; error 31 us; skew 500 ppm
Sep 29 03:05:26 dist-test-slave-bdps krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651126, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:26.628682 9088 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:05:26.629947 9088 webserver.cc:460] Webserver started at http://127.7.209.126:33017/ using document root <none> and password file <none>
I20260929 03:05:26.630725 9088 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:26.630801 9088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:26.631055 9088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:26.633653 9088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "772b38eb9f0e4a93969314ef309837d4"
format_stamp: "Formatted at 2026-09-29 03:05:26 on dist-test-slave-bdps"
server_key: "986909bed607d5d75b7b08a71b6003cd"
server_key_iv: "f90754e8b69982423e195a7be8ddc644"
server_key_version: "encryptionkey@0"
I20260929 03:05:26.634320 9088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "772b38eb9f0e4a93969314ef309837d4"
format_stamp: "Formatted at 2026-09-29 03:05:26 on dist-test-slave-bdps"
server_key: "986909bed607d5d75b7b08a71b6003cd"
server_key_iv: "f90754e8b69982423e195a7be8ddc644"
server_key_version: "encryptionkey@0"
I20260929 03:05:26.638832 9088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:05:26.641567 9103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:26.643047 9088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 03:05:26.643165 9088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "772b38eb9f0e4a93969314ef309837d4"
format_stamp: "Formatted at 2026-09-29 03:05:26 on dist-test-slave-bdps"
server_key: "986909bed607d5d75b7b08a71b6003cd"
server_key_iv: "f90754e8b69982423e195a7be8ddc644"
server_key_version: "encryptionkey@0"
I20260929 03:05:26.643271 9088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:26.661402 9088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:26.664691 9088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:26.664880 9088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:26.674149 9088 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:36121
I20260929 03:05:26.674170 9165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:36121 every 8 connection(s)
I20260929 03:05:26.675318 9088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:26.675993 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 9088
I20260929 03:05:26.676116 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:26.676424 8005 external_mini_cluster.cc:1428] Setting key b2432394fc2dfffd7151228d314a29e7
I20260929 03:05:26.679380 9166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 03:05:26 dist-test-slave-bdps krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:26.687513 9166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4: Bootstrap starting.
I20260929 03:05:26.690631 9166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:26.691985 9166 log.cc:824] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:26.692684 9169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:26.678244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35444 (local address 127.7.209.126:36121)
0929 03:05:26.678733 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:26.678757 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:26.678813 (+ 56us) server_negotiation.cc:407] Connection header received
0929 03:05:26.679815 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:26.679844 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:26.680352 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:26.680820 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:26.681578 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:26.683234 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:26.684137 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:26.684888 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:26.685069 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:26.687911 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:26.687917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:26.687934 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:26.687965 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:26.689676 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:26.690169 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:26.690174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:26.690179 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:26.690293 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:26.690559 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:26.690561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:26.690562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:26.691033 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:26.691184 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:26.691466 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 03:05:26.691787 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":132,"threads_started":1}
I20260929 03:05:26.694142 9166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4: No bootstrap required, opened a new log
I20260929 03:05:26.697329 9166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "772b38eb9f0e4a93969314ef309837d4" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 36121 } }
I20260929 03:05:26.697557 9166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:26.697633 9166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 772b38eb9f0e4a93969314ef309837d4, State: Initialized, Role: FOLLOWER
I20260929 03:05:26.698073 9166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "772b38eb9f0e4a93969314ef309837d4" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 36121 } }
I20260929 03:05:26.698220 9166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:26.698272 9166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:26.698346 9166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:26.699188 9166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "772b38eb9f0e4a93969314ef309837d4" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 36121 } }
I20260929 03:05:26.699555 9166 leader_election.cc:302] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 772b38eb9f0e4a93969314ef309837d4; no voters:
I20260929 03:05:26.699874 9166 leader_election.cc:288] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:26.699977 9171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:26.700202 9171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [term 1 LEADER]: Becoming Leader. State: Replica: 772b38eb9f0e4a93969314ef309837d4, State: Running, Role: LEADER
I20260929 03:05:26.700553 9171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "772b38eb9f0e4a93969314ef309837d4" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 36121 } }
I20260929 03:05:26.701102 9166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:26.702337 9172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "772b38eb9f0e4a93969314ef309837d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "772b38eb9f0e4a93969314ef309837d4" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 36121 } } }
I20260929 03:05:26.702461 9172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:26.702754 9173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 772b38eb9f0e4a93969314ef309837d4. Latest consensus state: current_term: 1 leader_uuid: "772b38eb9f0e4a93969314ef309837d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "772b38eb9f0e4a93969314ef309837d4" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 36121 } } }
I20260929 03:05:26.702844 9173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:26.702850 9180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:26.705960 9180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:26.711691 9180 catalog_manager.cc:1347] Generated new cluster ID: cf59f3adb6cf4de6bbbd3fdab971ca78
I20260929 03:05:26.711766 9180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:26.730180 9180 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:26.731249 9180 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:26.737458 9180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4: Generated new TSK 0
I20260929 03:05:26.738348 9180 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:05:26.767874 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:36121
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:26.895766 9192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:26.896059 9192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:26.896104 9192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:26.900188 9192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:26.900250 9192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:26.900288 9192 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 03:05:26.900328 9192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:26.906144 9192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.209.126:36121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:26.907507 9192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:26.908846 9192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:26.920553 9197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:26.920554 9200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:26.920704 9192 server_base.cc:1034] running on GCE node
W20260929 03:05:26.920980 9198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:26.921419 9192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:26.922184 9192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:26.923346 9192 hybrid_clock.cc:648] HybridClock initialized: now 1790651126923331 us; error 29 us; skew 500 ppm
Sep 29 03:05:26 dist-test-slave-bdps krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651126, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:26.926357 9192 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:05:26.927407 9192 webserver.cc:460] Webserver started at http://127.7.209.65:45633/ using document root <none> and password file <none>
I20260929 03:05:26.928033 9192 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:26.928095 9192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:26.928295 9192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:26.930743 9192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "e70590ac886b4d859494447ab01251ff"
format_stamp: "Formatted at 2026-09-29 03:05:26 on dist-test-slave-bdps"
server_key: "778d34f65777c5f6a22d19d21ec2d0b0"
server_key_iv: "03070c3906bd9895ebb4a06bf7f9de07"
server_key_version: "encryptionkey@0"
I20260929 03:05:26.931339 9192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "e70590ac886b4d859494447ab01251ff"
format_stamp: "Formatted at 2026-09-29 03:05:26 on dist-test-slave-bdps"
server_key: "778d34f65777c5f6a22d19d21ec2d0b0"
server_key_iv: "03070c3906bd9895ebb4a06bf7f9de07"
server_key_version: "encryptionkey@0"
I20260929 03:05:26.935493 9192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:05:26.938073 9207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:26.939392 9192 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 03:05:26.939488 9192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "e70590ac886b4d859494447ab01251ff"
format_stamp: "Formatted at 2026-09-29 03:05:26 on dist-test-slave-bdps"
server_key: "778d34f65777c5f6a22d19d21ec2d0b0"
server_key_iv: "03070c3906bd9895ebb4a06bf7f9de07"
server_key_version: "encryptionkey@0"
I20260929 03:05:26.939590 9192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:26.953272 9192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:26.956702 9192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:26.956881 9192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:26.958719 9192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:26.958773 9192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:26.958819 9192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:26.958848 9192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 03:05:26 dist-test-slave-bdps krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651126, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:26.971413 9192 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:38627
I20260929 03:05:26.971462 9322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:38627 every 8 connection(s)
I20260929 03:05:26.972580 9192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:26.974041 9169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:26.959738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:50429 (local address 127.7.209.126:36121)
0929 03:05:26.959875 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:26.959879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:26.961184 (+ 1305us) server_negotiation.cc:407] Connection header received
0929 03:05:26.962486 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:26.962491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:26.962567 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:26.962679 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:26.964060 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:26.964662 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:26.965967 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:26.966236 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:26.966329 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:26.969418 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:26.969423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:26.969427 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:26.969460 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:26.971145 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:26.971809 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:26.971813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:26.971815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:26.971917 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:26.972235 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:26.972238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:26.972240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:26.972475 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:26.972579 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:26.973735 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0929 03:05:26.973879 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:05:26.974478 9216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:26.960410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:36121 (local address 127.7.209.65:50429)
0929 03:05:26.961015 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:26.961062 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:26.962199 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:26.962899 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:26.962907 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:26.963296 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:26.963902 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:26.963917 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:26.964784 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:26.964788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:26.965841 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:26.965850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:26.966455 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:26.966458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:26.966621 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:26.967212 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:26.967229 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:26.969257 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:26.971258 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:26.971264 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:26.971281 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:26.971704 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:26.972000 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:26.972002 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:26.972004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:26.972147 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:26.972563 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:26.972567 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:26.972848 (+ 281us) client_negotiation.cc:769] Sending connection context
0929 03:05:26.973041 (+ 193us) client_negotiation.cc:240] Negotiation successful
0929 03:05:26.973396 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":165,"threads_started":1}
I20260929 03:05:26.975279 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 9192
I20260929 03:05:26.975406 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:05:26.975741 8005 external_mini_cluster.cc:1428] Setting key 5da71edc7d5defdc880733f834e8fa9a
I20260929 03:05:26.975761 9323 heartbeater.cc:347] Connected to a master server at 127.7.209.126:36121
I20260929 03:05:26.976001 9323 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:26.976615 9323 heartbeater.cc:511] Master 127.7.209.126:36121 requested a full tablet report, sending...
I20260929 03:05:26.978391 9119 ts_manager.cc:194] Registered new tserver with Master: e70590ac886b4d859494447ab01251ff (127.7.209.65:38627)
I20260929 03:05:26.980146 9119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:50429
I20260929 03:05:26.986864 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:26.978799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:36121 (local address 127.7.209.65:33455)
0929 03:05:26.979121 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:26.979136 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:26.979235 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:26.979466 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:26.979470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:26.979685 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:26.979814 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:26.979820 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:26.980835 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:26.980838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:26.981661 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:26.981672 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:26.982231 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:26.982234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:26.982329 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:26.982881 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:26.982895 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:26.983366 (+ 471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:26.985274 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:26.985277 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:26.985278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:26.985489 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:26.985832 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:26.985834 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:26.985835 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:26.985918 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:26.986362 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:26.986364 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:26.986447 (+ 83us) client_negotiation.cc:769] Sending connection context
0929 03:05:26.986526 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 03:05:26.986677 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":103,"threads_started":1}
I20260929 03:05:26.986936 9169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:26.978869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:33455 (local address 127.7.209.126:36121)
0929 03:05:26.978984 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:26.978988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:26.979209 (+ 221us) server_negotiation.cc:407] Connection header received
0929 03:05:26.979311 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:26.979313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:26.979359 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:26.979441 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:26.979917 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:26.980707 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:26.981779 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:26.982100 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:26.982209 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:26.983521 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:26.983525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:26.983528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:26.983554 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:26.985146 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:26.985593 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:26.985597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:26.985599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:26.985742 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:26.986002 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:26.986005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:26.986007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:26.986265 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:26.986386 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:26.986610 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 03:05:26.986760 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:05:27.006181 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:36121
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:27.131098 9332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:27.131407 9332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:27.131449 9332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:27.135918 9332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:27.135990 9332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:27.136029 9332 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 03:05:27.136062 9332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:27.141518 9332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.209.126:36121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:27.142863 9332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:27.144167 9332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:27.155272 9340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:27.155277 9337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:27.155274 9338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:27.155663 9332 server_base.cc:1034] running on GCE node
I20260929 03:05:27.156107 9332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:27.156804 9332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:27.157948 9332 hybrid_clock.cc:648] HybridClock initialized: now 1790651127157920 us; error 44 us; skew 500 ppm
Sep 29 03:05:27 dist-test-slave-bdps krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:27.161177 9332 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:05:27.162381 9332 webserver.cc:460] Webserver started at http://127.7.209.66:40669/ using document root <none> and password file <none>
I20260929 03:05:27.163071 9332 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:27.163144 9332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:27.163370 9332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:27.165860 9332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "c1974404eef64b4eaef213561a437203"
format_stamp: "Formatted at 2026-09-29 03:05:27 on dist-test-slave-bdps"
server_key: "5dae379413ef067f4a838a102655a931"
server_key_iv: "e01eae30accc24b1ed7135d26aaafb45"
server_key_version: "encryptionkey@0"
I20260929 03:05:27.166488 9332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "c1974404eef64b4eaef213561a437203"
format_stamp: "Formatted at 2026-09-29 03:05:27 on dist-test-slave-bdps"
server_key: "5dae379413ef067f4a838a102655a931"
server_key_iv: "e01eae30accc24b1ed7135d26aaafb45"
server_key_version: "encryptionkey@0"
I20260929 03:05:27.170892 9332 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:27.173566 9347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:27.175020 9332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:27.175130 9332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "c1974404eef64b4eaef213561a437203"
format_stamp: "Formatted at 2026-09-29 03:05:27 on dist-test-slave-bdps"
server_key: "5dae379413ef067f4a838a102655a931"
server_key_iv: "e01eae30accc24b1ed7135d26aaafb45"
server_key_version: "encryptionkey@0"
I20260929 03:05:27.175231 9332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:27.190722 9332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:27.194957 9332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:27.195190 9332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:27.197060 9332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:27.197118 9332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:27.197187 9332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:27.197218 9332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 03:05:27 dist-test-slave-bdps krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:27.209892 9332 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:38235
I20260929 03:05:27.209910 9462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:38235 every 8 connection(s)
I20260929 03:05:27.210958 9332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:27.213215 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 9332
I20260929 03:05:27.213331 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:05:27.213696 8005 external_mini_cluster.cc:1428] Setting key 77841dbe39c52c5560a9a03a0c7f831b
I20260929 03:05:27.214098 9169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.198054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:44589 (local address 127.7.209.126:36121)
0929 03:05:27.198196 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:27.198200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:27.199615 (+ 1415us) server_negotiation.cc:407] Connection header received
0929 03:05:27.200745 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:27.200750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:27.200821 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:27.200874 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:27.202942 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.203831 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.205675 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.206036 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.206162 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.210144 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:27.210147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:27.210150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:27.210179 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:27.211806 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:27.212312 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:27.212316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:27.212318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:27.212410 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:27.212753 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:27.212756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:27.212757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:27.212998 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:27.213117 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:27.213754 (+ 637us) server_negotiation.cc:299] Negotiation successful
0929 03:05:27.213897 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:05:27.215138 9356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.198778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:36121 (local address 127.7.209.66:44589)
0929 03:05:27.199396 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:27.199446 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:27.200460 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:27.201369 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:27.201379 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:27.201942 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:27.202770 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.202794 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.204011 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.204015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.205509 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.205521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.206214 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.206218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.206427 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.207767 (+ 1340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:27.207792 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:27.209958 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:27.211915 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:27.211922 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:27.211939 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:27.212199 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:27.212495 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:27.212498 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:27.212501 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:27.212653 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:27.213124 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:27.213130 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:27.213456 (+ 326us) client_negotiation.cc:769] Sending connection context
0929 03:05:27.213685 (+ 229us) client_negotiation.cc:240] Negotiation successful
0929 03:05:27.214003 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":164,"threads_started":1}
I20260929 03:05:27.217434 9463 heartbeater.cc:347] Connected to a master server at 127.7.209.126:36121
I20260929 03:05:27.217654 9463 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:27.218348 9463 heartbeater.cc:511] Master 127.7.209.126:36121 requested a full tablet report, sending...
I20260929 03:05:27.219710 9120 ts_manager.cc:194] Registered new tserver with Master: c1974404eef64b4eaef213561a437203 (127.7.209.66:38235)
I20260929 03:05:27.220616 9120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:44589
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:05:27.227815 9169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.219316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:51829 (local address 127.7.209.126:36121)
0929 03:05:27.219442 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:27.219446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:27.219798 (+ 352us) server_negotiation.cc:407] Connection header received
0929 03:05:27.219968 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:27.219972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:27.220025 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:27.220114 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:27.221317 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.221952 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.222913 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.223154 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.223249 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.224625 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:27.224651 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:27.224653 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:27.224677 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:27.226086 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:27.226488 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:27.226492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:27.226495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:27.226574 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:27.226847 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:27.226850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:27.226851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:27.227153 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:27.227246 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:27.227490 (+ 244us) server_negotiation.cc:299] Negotiation successful
0929 03:05:27.227628 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 03:05:27.227835 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.219440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:36121 (local address 127.7.209.66:51829)
0929 03:05:27.219714 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:27.219729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:27.219864 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:27.220147 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:27.220150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:27.220387 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:27.220517 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.220524 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.222095 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.222099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.222800 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.222809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.223266 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.223270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.223403 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.224066 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:27.224078 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:27.224499 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:27.226191 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:27.226194 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:27.226197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:27.226401 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:27.226656 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:27.226660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:27.226662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:27.226753 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:27.227242 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:27.227245 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:27.227340 (+ 95us) client_negotiation.cc:769] Sending connection context
0929 03:05:27.227433 (+ 93us) client_negotiation.cc:240] Negotiation successful
0929 03:05:27.227615 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
W20260929 03:05:27.588565 8005 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.209.67: real 0.373s user 0.000s sys 0.005s
I20260929 03:05:27.589216 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:36121
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:27.719986 9472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:27.720327 9472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:27.720400 9472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:27.724615 9472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:27.724695 9472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:27.724754 9472 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 03:05:27.724810 9472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:27.730679 9472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.209.126:36121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:27.731983 9472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:27.733244 9472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:27.744907 9478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:27.745007 9472 server_base.cc:1034] running on GCE node
W20260929 03:05:27.744938 9480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:27.745249 9477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:27.745760 9472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:27.746578 9472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:27.747740 9472 hybrid_clock.cc:648] HybridClock initialized: now 1790651127747719 us; error 36 us; skew 500 ppm
Sep 29 03:05:27 dist-test-slave-bdps krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:27.751582 9472 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:05:27.752862 9472 webserver.cc:460] Webserver started at http://127.7.209.67:39957/ using document root <none> and password file <none>
I20260929 03:05:27.753569 9472 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:27.753679 9472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:27.753940 9472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:27.756837 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "21a13d6250e14739882c2dfea62b75b5"
format_stamp: "Formatted at 2026-09-29 03:05:27 on dist-test-slave-bdps"
server_key: "48d02a34a53d58027a25a3801689befa"
server_key_iv: "20d4f09f84954598235e104ab7095125"
server_key_version: "encryptionkey@0"
I20260929 03:05:27.757517 9472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "21a13d6250e14739882c2dfea62b75b5"
format_stamp: "Formatted at 2026-09-29 03:05:27 on dist-test-slave-bdps"
server_key: "48d02a34a53d58027a25a3801689befa"
server_key_iv: "20d4f09f84954598235e104ab7095125"
server_key_version: "encryptionkey@0"
I20260929 03:05:27.762071 9472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:27.764878 9487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:27.766341 9472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:27.766449 9472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "21a13d6250e14739882c2dfea62b75b5"
format_stamp: "Formatted at 2026-09-29 03:05:27 on dist-test-slave-bdps"
server_key: "48d02a34a53d58027a25a3801689befa"
server_key_iv: "20d4f09f84954598235e104ab7095125"
server_key_version: "encryptionkey@0"
I20260929 03:05:27.766548 9472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:27.791374 9472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:27.795372 9472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:27.795584 9472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:27.797338 9472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:27.797389 9472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:27.797435 9472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:27.797470 9472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:27.809162 9472 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:40063
Sep 29 03:05:27 dist-test-slave-bdps krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651127, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:27.809190 9603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:40063 every 8 connection(s)
I20260929 03:05:27.810389 9472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:27.813575 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.798633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:45873 (local address 127.7.209.126:36121)
0929 03:05:27.798987 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:27.798992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:27.799932 (+ 940us) server_negotiation.cc:407] Connection header received
0929 03:05:27.801469 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:27.801474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:27.801552 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:27.801718 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:27.803446 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.804390 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.806207 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.806593 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.806719 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.809967 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:27.809971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:27.809973 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:27.810002 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:27.811719 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:27.812206 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:27.812208 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:27.812210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:27.812295 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:27.812555 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:27.812557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:27.812558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:27.812767 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:27.812842 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:27.813308 (+ 466us) server_negotiation.cc:299] Negotiation successful
0929 03:05:27.813425 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":144,"threads_started":1}
I20260929 03:05:27.814518 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.799154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:36121 (local address 127.7.209.67:45873)
0929 03:05:27.799745 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:27.799803 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:27.801202 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:27.801927 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:27.801936 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:27.802450 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:27.803244 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.803268 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.804538 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.804541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.806071 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.806080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.806785 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.806788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.806997 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.807810 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:27.807834 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:27.809778 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:27.811830 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:27.811835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:27.811848 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:27.812128 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:27.812359 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:27.812361 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:27.812362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:27.812492 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:27.812841 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:27.812847 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:27.813110 (+ 263us) client_negotiation.cc:769] Sending connection context
0929 03:05:27.813275 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 03:05:27.813552 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":123,"threads_started":1}
I20260929 03:05:27.815929 9604 heartbeater.cc:347] Connected to a master server at 127.7.209.126:36121
I20260929 03:05:27.816066 9604 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:27.816596 9604 heartbeater.cc:511] Master 127.7.209.126:36121 requested a full tablet report, sending...
I20260929 03:05:27.817284 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 9472
I20260929 03:05:27.817431 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:05:27.817759 9119 ts_manager.cc:194] Registered new tserver with Master: 21a13d6250e14739882c2dfea62b75b5 (127.7.209.67:40063)
I20260929 03:05:27.817836 8005 external_mini_cluster.cc:1428] Setting key 62fa001e8f177228500f89aa3ca394d0
I20260929 03:05:27.818513 9119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:45873
I20260929 03:05:27.820052 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:05:27.828311 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.818621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:36121 (local address 127.7.209.67:44713)
0929 03:05:27.819074 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:27.819089 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:27.819191 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:27.819583 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:27.819585 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:27.819810 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:27.819937 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.819945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.822667 (+ 2722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.822671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.823508 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.823520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.823978 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.823981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.824074 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.824592 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:27.824605 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:27.825041 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:27.826854 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:27.826857 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:27.826859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:27.827048 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:27.827338 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:27.827340 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:27.827341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:27.827411 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:27.827845 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:27.827847 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:27.827925 (+ 78us) client_negotiation.cc:769] Sending connection context
0929 03:05:27.828001 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 03:05:27.828119 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":154,"threads_started":1}
I20260929 03:05:27.828398 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.818497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:44713 (local address 127.7.209.126:36121)
0929 03:05:27.818615 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:27.818619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:27.819153 (+ 534us) server_negotiation.cc:407] Connection header received
0929 03:05:27.819330 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:27.819333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:27.819380 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:27.819427 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:27.821770 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.822490 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.823634 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.823873 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.823956 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.825166 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:27.825169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:27.825172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:27.825193 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:27.826749 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:27.827146 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:27.827149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:27.827151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:27.827250 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:27.827478 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:27.827481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:27.827483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:27.827761 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:27.827865 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:27.828067 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 03:05:27.828225 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Sep 29 03:05:27 dist-test-slave-bdps krb5kdc[9075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651127, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 03:05:27 dist-test-slave-bdps krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651127, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:27.848941 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.840851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47536 (local address 127.7.209.126:36121)
0929 03:05:27.841006 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:27.841011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:27.841049 (+ 38us) server_negotiation.cc:407] Connection header received
0929 03:05:27.841210 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:27.841212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:27.841259 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:27.841332 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:27.842055 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.842650 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.843497 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.843726 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.843814 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.846234 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:27.846237 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:27.846239 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:27.846265 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:27.847477 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:27.847877 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:27.847880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:27.847881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:27.847961 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:27.848225 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:27.848227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:27.848229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:27.848407 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:27.848518 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:27.848697 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 03:05:27.848798 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:05:27.852028 9119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:05:27.855748 9119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:27.872092 9624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.865487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:38235 (local address 127.0.0.1:55690)
0929 03:05:27.866744 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:27.866757 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:27.866864 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:27.867390 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:27.867396 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:27.867418 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:27.867587 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.867595 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.869038 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.869041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.870169 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.870179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.871519 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.871523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.871610 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.871642 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 03:05:27.871728 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 03:05:27.871897 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1198,"mutex_wait_us":57,"thread_start_us":164,"threads_started":1}
I20260929 03:05:27.872267 9626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.866091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52740 (local address 127.7.209.65:38627)
0929 03:05:27.866620 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:27.866627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:27.866726 (+ 99us) server_negotiation.cc:407] Connection header received
0929 03:05:27.866992 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:27.866999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:27.867247 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:27.867400 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:27.867824 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.869229 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.870648 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.871330 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.871435 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.871610 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:27.871869 (+ 259us) server_negotiation.cc:299] Negotiation successful
0929 03:05:27.872100 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":104,"threads_started":1}
I20260929 03:05:27.872460 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.866513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40808 (local address 127.7.209.67:40063)
0929 03:05:27.867013 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:27.867026 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:27.867115 (+ 89us) server_negotiation.cc:407] Connection header received
0929 03:05:27.867223 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:27.867228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:27.867390 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:27.867530 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:27.867904 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.868806 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.870594 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.871459 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.871566 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.871728 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:27.872180 (+ 452us) server_negotiation.cc:299] Negotiation successful
0929 03:05:27.872307 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":142,"threads_started":1}
I20260929 03:05:27.872700 9623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.865664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:40063 (local address 127.0.0.1:40808)
0929 03:05:27.867023 (+ 1359us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:27.867035 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:27.867118 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:27.867694 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:27.867697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:27.867711 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:27.867800 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.867805 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.868905 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.868908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.870473 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.870483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.871782 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.871786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.871874 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.871892 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 03:05:27.872015 (+ 123us) client_negotiation.cc:240] Negotiation successful
0929 03:05:27.872079 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1308,"mutex_wait_us":40,"thread_start_us":94,"threads_started":1}
I20260929 03:05:27.872807 9625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.865865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55690 (local address 127.7.209.66:38235)
0929 03:05:27.866305 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:27.866314 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:27.866868 (+ 554us) server_negotiation.cc:407] Connection header received
0929 03:05:27.866973 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:27.866979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:27.867249 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:27.867400 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:27.867695 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.868905 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.870309 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.871143 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.871250 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.871446 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:27.872536 (+ 1090us) server_negotiation.cc:299] Negotiation successful
0929 03:05:27.872672 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":83,"threads_started":1}
I20260929 03:05:27.873143 9622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.865355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:38627 (local address 127.0.0.1:52740)
0929 03:05:27.866555 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:27.866615 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:27.866775 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:27.867574 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:27.867577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:27.867594 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:27.867698 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.867705 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.869344 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.869348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.870548 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.870556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.871432 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.871436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.871569 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.871609 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 03:05:27.871745 (+ 136us) client_negotiation.cc:240] Negotiation successful
0929 03:05:27.873011 (+ 1266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"mutex_wait_us":1133,"thread_start_us":127,"threads_started":1}
I20260929 03:05:27.876128 9397 tablet_service.cc:1467] Processing CreateTablet for tablet 4a75ffd5d7c74f1aa5303a66e247f83f (DEFAULT_TABLE table=test-table [id=5f043ca28eac49f4a0352589096b08b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:27.876137 9257 tablet_service.cc:1467] Processing CreateTablet for tablet 4a75ffd5d7c74f1aa5303a66e247f83f (DEFAULT_TABLE table=test-table [id=5f043ca28eac49f4a0352589096b08b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:27.876132 9538 tablet_service.cc:1467] Processing CreateTablet for tablet 4a75ffd5d7c74f1aa5303a66e247f83f (DEFAULT_TABLE table=test-table [id=5f043ca28eac49f4a0352589096b08b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:27.877360 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a75ffd5d7c74f1aa5303a66e247f83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:27.877364 9257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a75ffd5d7c74f1aa5303a66e247f83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:27.877372 9538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a75ffd5d7c74f1aa5303a66e247f83f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:27.884512 9628 tablet_bootstrap.cc:492] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203: Bootstrap starting.
I20260929 03:05:27.886008 9630 tablet_bootstrap.cc:492] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5: Bootstrap starting.
I20260929 03:05:27.886070 9629 tablet_bootstrap.cc:492] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff: Bootstrap starting.
I20260929 03:05:27.887702 9628 tablet_bootstrap.cc:654] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:27.888105 9630 tablet_bootstrap.cc:654] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:27.888502 9629 tablet_bootstrap.cc:654] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:27.888690 9628 log.cc:824] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:27.888871 9630 log.cc:824] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:27.889339 9629 log.cc:824] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:27.890914 9628 tablet_bootstrap.cc:492] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203: No bootstrap required, opened a new log
I20260929 03:05:27.891144 9628 ts_tablet_manager.cc:1397] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260929 03:05:27.891296 9630 tablet_bootstrap.cc:492] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5: No bootstrap required, opened a new log
I20260929 03:05:27.891297 9629 tablet_bootstrap.cc:492] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff: No bootstrap required, opened a new log
I20260929 03:05:27.891480 9629 ts_tablet_manager.cc:1397] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260929 03:05:27.891480 9630 ts_tablet_manager.cc:1397] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260929 03:05:27.894862 9630 raft_consensus.cc:348] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.895116 9630 raft_consensus.cc:374] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:27.895186 9630 raft_consensus.cc:729] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a13d6250e14739882c2dfea62b75b5, State: Initialized, Role: FOLLOWER
I20260929 03:05:27.895701 9630 consensus_queue.cc:260] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.896047 9628 raft_consensus.cc:348] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.896047 9629 raft_consensus.cc:348] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.896322 9629 raft_consensus.cc:374] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:27.896381 9628 raft_consensus.cc:374] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:27.896412 9629 raft_consensus.cc:729] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70590ac886b4d859494447ab01251ff, State: Initialized, Role: FOLLOWER
I20260929 03:05:27.896415 9604 heartbeater.cc:503] Master 127.7.209.126:36121 was elected leader, sending a full tablet report...
I20260929 03:05:27.896471 9628 raft_consensus.cc:729] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1974404eef64b4eaef213561a437203, State: Initialized, Role: FOLLOWER
I20260929 03:05:27.896847 9630 ts_tablet_manager.cc:1428] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 03:05:27.896973 9628 consensus_queue.cc:260] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.896970 9629 consensus_queue.cc:260] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.897702 9323 heartbeater.cc:503] Master 127.7.209.126:36121 was elected leader, sending a full tablet report...
I20260929 03:05:27.898110 9628 ts_tablet_manager.cc:1428] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260929 03:05:27.898485 9463 heartbeater.cc:503] Master 127.7.209.126:36121 was elected leader, sending a full tablet report...
I20260929 03:05:27.899076 9629 ts_tablet_manager.cc:1428] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff: Time spent starting tablet: real 0.008s user 0.005s sys 0.002s
I20260929 03:05:27.899749 9636 raft_consensus.cc:482] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:27.899982 9636 raft_consensus.cc:504] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.902976 9636 leader_election.cc:288] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e70590ac886b4d859494447ab01251ff (127.7.209.65:38627), 21a13d6250e14739882c2dfea62b75b5 (127.7.209.67:40063)
I20260929 03:05:27.906214 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.903069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:37865 (local address 127.7.209.67:40063)
0929 03:05:27.903192 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:27.903196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:27.903208 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:05:27.903249 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:27.903253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:27.903297 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:27.903370 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:27.903619 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.904174 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.905225 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.905687 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.905792 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.905855 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:27.905989 (+ 134us) server_negotiation.cc:299] Negotiation successful
0929 03:05:27.906063 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 03:05:27.906231 9626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.903238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:54507 (local address 127.7.209.65:38627)
0929 03:05:27.903357 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:27.903361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:27.903374 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:05:27.903413 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:27.903415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:27.903452 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:27.903514 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:27.903747 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.904238 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.905234 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.905721 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.905823 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.905877 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:27.906041 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 03:05:27.906118 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 03:05:27.906243 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.902720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:40063 (local address 127.7.209.66:37865)
0929 03:05:27.902999 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:27.903014 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:27.903135 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:27.903383 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:27.903385 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:27.903400 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:27.903513 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.903518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.904272 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.904275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.905114 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.905123 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.905785 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.905789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.905879 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.905903 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 03:05:27.905968 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 03:05:27.906039 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":21,"thread_start_us":122,"threads_started":1}
I20260929 03:05:27.906255 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.902694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:38627 (local address 127.7.209.66:54507)
0929 03:05:27.902987 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:27.903005 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:27.903113 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:27.903526 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:27.903528 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:27.903544 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:27.903657 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.903662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.904332 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.904335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.905121 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:27.905129 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.905818 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.905822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:27.905926 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.905947 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 03:05:27.906008 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 03:05:27.906072 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260929 03:05:27.906807 9558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a75ffd5d7c74f1aa5303a66e247f83f" candidate_uuid: "c1974404eef64b4eaef213561a437203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a13d6250e14739882c2dfea62b75b5" is_pre_election: true
I20260929 03:05:27.906924 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a75ffd5d7c74f1aa5303a66e247f83f" candidate_uuid: "c1974404eef64b4eaef213561a437203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70590ac886b4d859494447ab01251ff" is_pre_election: true
I20260929 03:05:27.907100 9558 raft_consensus.cc:2454] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1974404eef64b4eaef213561a437203 in term 0.
I20260929 03:05:27.907178 9277 raft_consensus.cc:2454] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1974404eef64b4eaef213561a437203 in term 0.
I20260929 03:05:27.907608 9351 leader_election.cc:302] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1974404eef64b4eaef213561a437203, e70590ac886b4d859494447ab01251ff; no voters:
I20260929 03:05:27.907902 9636 raft_consensus.cc:2781] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:27.907979 9636 raft_consensus.cc:482] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:27.908032 9636 raft_consensus.cc:3037] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:27.909500 9636 raft_consensus.cc:504] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.909875 9636 leader_election.cc:288] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [CANDIDATE]: Term 1 election: Requested vote from peers e70590ac886b4d859494447ab01251ff (127.7.209.65:38627), 21a13d6250e14739882c2dfea62b75b5 (127.7.209.67:40063)
I20260929 03:05:27.910213 9558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a75ffd5d7c74f1aa5303a66e247f83f" candidate_uuid: "c1974404eef64b4eaef213561a437203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a13d6250e14739882c2dfea62b75b5"
I20260929 03:05:27.910347 9558 raft_consensus.cc:3037] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:27.911579 9558 raft_consensus.cc:2454] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1974404eef64b4eaef213561a437203 in term 1.
I20260929 03:05:27.911901 9349 leader_election.cc:302] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a13d6250e14739882c2dfea62b75b5, c1974404eef64b4eaef213561a437203; no voters:
I20260929 03:05:27.911870 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a75ffd5d7c74f1aa5303a66e247f83f" candidate_uuid: "c1974404eef64b4eaef213561a437203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70590ac886b4d859494447ab01251ff"
I20260929 03:05:27.912010 9277 raft_consensus.cc:3037] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:27.913143 9277 raft_consensus.cc:2454] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1974404eef64b4eaef213561a437203 in term 1.
I20260929 03:05:27.913306 9636 raft_consensus.cc:2781] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:27.913582 9636 raft_consensus.cc:686] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 1 LEADER]: Becoming Leader. State: Replica: c1974404eef64b4eaef213561a437203, State: Running, Role: LEADER
I20260929 03:05:27.913968 9636 consensus_queue.cc:237] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.917806 9120 catalog_manager.cc:5495] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1974404eef64b4eaef213561a437203 (127.7.209.66). New cstate: current_term: 1 leader_uuid: "c1974404eef64b4eaef213561a437203" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } health_report { overall_health: UNKNOWN } } }
I20260929 03:05:27.946594 9625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:27.942891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55692 (local address 127.7.209.66:38235)
0929 03:05:27.943012 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:27.943017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:27.943030 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:05:27.943226 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:27.943228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:27.943293 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:27.943380 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:27.943671 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.944241 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.945092 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:27.945318 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:27.945417 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:27.945764 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:27.946241 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:27.946331 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:27.946385 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 03:05:27.946453 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 03:05:27.953776 9277 raft_consensus.cc:1261] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff [term 1 FOLLOWER]: Refusing update from remote peer c1974404eef64b4eaef213561a437203: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:27.954133 9558 raft_consensus.cc:1261] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5 [term 1 FOLLOWER]: Refusing update from remote peer c1974404eef64b4eaef213561a437203: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:27.954598 9639 consensus_queue.cc:1035] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:27.954802 9636 consensus_queue.cc:1035] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 03:05:27.962580 9464 tablet.cc:2369] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:05:27.964562 9644 mvcc.cc:205] Tried to move back new op lower bound from 7334507020089618432 to 7334507019938496512. Current Snapshot: MvccSnapshot[applied={T|T < 7334507020089618432}]
I20260929 03:05:27.965031 9643 mvcc.cc:205] Tried to move back new op lower bound from 7334507020089618432 to 7334507019938496512. Current Snapshot: MvccSnapshot[applied={T|T < 7334507020089618432}]
I20260929 03:05:27.977037 9120 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:27.977233 9120 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:27.979995 9120 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4: Sending DeleteTablet for 3 replicas of tablet 4a75ffd5d7c74f1aa5303a66e247f83f
I20260929 03:05:27.980804 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a75ffd5d7c74f1aa5303a66e247f83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:27 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:55690
I20260929 03:05:27.980849 9257 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a75ffd5d7c74f1aa5303a66e247f83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:27 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:52740
I20260929 03:05:27.981499 9653 tablet_replica.cc:330] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff: stopping tablet replica
I20260929 03:05:27.981526 9538 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a75ffd5d7c74f1aa5303a66e247f83f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:27 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:40808
I20260929 03:05:27.981844 9653 raft_consensus.cc:2229] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:27.982155 9653 raft_consensus.cc:2258] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:27.982470 9652 tablet_replica.cc:330] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203: stopping tablet replica
I20260929 03:05:27.982741 9652 raft_consensus.cc:2229] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:05:27.982842 9120 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 03:05:27.983181 9652 raft_consensus.cc:2258] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 03:05:27.983233 9120 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:27.983661 9653 ts_tablet_manager.cc:1905] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:27.984457 9652 ts_tablet_manager.cc:1905] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:27.986649 9653 ts_tablet_manager.cc:1918] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:27.986809 9653 log.cc:1196] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/wal/wals/4a75ffd5d7c74f1aa5303a66e247f83f
I20260929 03:05:27.987259 9653 ts_tablet_manager.cc:1939] T 4a75ffd5d7c74f1aa5303a66e247f83f P e70590ac886b4d859494447ab01251ff: Deleting consensus metadata
I20260929 03:05:27.987676 9652 ts_tablet_manager.cc:1918] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:27.987972 9654 tablet_replica.cc:330] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5: stopping tablet replica
I20260929 03:05:27.988368 9654 raft_consensus.cc:2229] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:27.988396 9257 tablet_service.cc:1467] Processing CreateTablet for tablet f3cc6e14336049e19d4807ce59506ad6 (DEFAULT_TABLE table=test-table [id=f3ac5343afb149e1a6519e0c85a44466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:27.988691 9257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3cc6e14336049e19d4807ce59506ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:27.988703 9107 catalog_manager.cc:4841] TS e70590ac886b4d859494447ab01251ff (127.7.209.65:38627): tablet 4a75ffd5d7c74f1aa5303a66e247f83f (table test-table [id=5f043ca28eac49f4a0352589096b08b7]) successfully deleted
I20260929 03:05:27.988801 9654 raft_consensus.cc:2258] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:27.988996 9652 log.cc:1196] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/4a75ffd5d7c74f1aa5303a66e247f83f
I20260929 03:05:27.989483 9652 ts_tablet_manager.cc:1939] T 4a75ffd5d7c74f1aa5303a66e247f83f P c1974404eef64b4eaef213561a437203: Deleting consensus metadata
I20260929 03:05:27.990504 9654 ts_tablet_manager.cc:1905] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:27.991215 9629 tablet_bootstrap.cc:492] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff: Bootstrap starting.
I20260929 03:05:27.991626 9104 catalog_manager.cc:4841] TS c1974404eef64b4eaef213561a437203 (127.7.209.66:38235): tablet 4a75ffd5d7c74f1aa5303a66e247f83f (table test-table [id=5f043ca28eac49f4a0352589096b08b7]) successfully deleted
I20260929 03:05:27.992157 9629 tablet_bootstrap.cc:654] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:27.992399 9397 tablet_service.cc:1467] Processing CreateTablet for tablet f3cc6e14336049e19d4807ce59506ad6 (DEFAULT_TABLE table=test-table [id=f3ac5343afb149e1a6519e0c85a44466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:27.992658 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3cc6e14336049e19d4807ce59506ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:27.993582 9538 tablet_service.cc:1467] Processing CreateTablet for tablet f3cc6e14336049e19d4807ce59506ad6 (DEFAULT_TABLE table=test-table [id=f3ac5343afb149e1a6519e0c85a44466]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:27.993678 9654 ts_tablet_manager.cc:1918] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:27.993775 9654 log.cc:1196] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/4a75ffd5d7c74f1aa5303a66e247f83f
I20260929 03:05:27.993875 9538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3cc6e14336049e19d4807ce59506ad6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:27.994172 9654 ts_tablet_manager.cc:1939] T 4a75ffd5d7c74f1aa5303a66e247f83f P 21a13d6250e14739882c2dfea62b75b5: Deleting consensus metadata
I20260929 03:05:27.995112 9628 tablet_bootstrap.cc:492] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203: Bootstrap starting.
I20260929 03:05:27.995376 9105 catalog_manager.cc:4841] TS 21a13d6250e14739882c2dfea62b75b5 (127.7.209.67:40063): tablet 4a75ffd5d7c74f1aa5303a66e247f83f (table test-table [id=5f043ca28eac49f4a0352589096b08b7]) successfully deleted
I20260929 03:05:27.996244 9630 tablet_bootstrap.cc:492] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5: Bootstrap starting.
I20260929 03:05:27.996695 9628 tablet_bootstrap.cc:654] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:27.997186 9630 tablet_bootstrap.cc:654] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:27.998421 9628 tablet_bootstrap.cc:492] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203: No bootstrap required, opened a new log
I20260929 03:05:27.998453 9629 tablet_bootstrap.cc:492] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff: No bootstrap required, opened a new log
I20260929 03:05:27.998502 9628 ts_tablet_manager.cc:1397] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20260929 03:05:27.998536 9629 ts_tablet_manager.cc:1397] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s
I20260929 03:05:27.998682 9630 tablet_bootstrap.cc:492] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5: No bootstrap required, opened a new log
I20260929 03:05:27.998755 9630 ts_tablet_manager.cc:1397] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 03:05:27.999173 9628 raft_consensus.cc:348] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.999289 9628 raft_consensus.cc:374] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:27.999318 9628 raft_consensus.cc:729] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1974404eef64b4eaef213561a437203, State: Initialized, Role: FOLLOWER
I20260929 03:05:27.999264 9629 raft_consensus.cc:348] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.999370 9629 raft_consensus.cc:374] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:27.999401 9629 raft_consensus.cc:729] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70590ac886b4d859494447ab01251ff, State: Initialized, Role: FOLLOWER
I20260929 03:05:27.999362 9630 raft_consensus.cc:348] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.999457 9630 raft_consensus.cc:374] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:27.999436 9628 consensus_queue.cc:260] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.999490 9630 raft_consensus.cc:729] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a13d6250e14739882c2dfea62b75b5, State: Initialized, Role: FOLLOWER
I20260929 03:05:27.999516 9629 consensus_queue.cc:260] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.999631 9630 consensus_queue.cc:260] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:27.999696 9628 ts_tablet_manager.cc:1428] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:27.999763 9629 ts_tablet_manager.cc:1428] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260929 03:05:27.999861 9630 ts_tablet_manager.cc:1428] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20260929 03:05:28.061972 9605 tablet.cc:2369] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:05:28.225159 9324 tablet.cc:2369] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:05:28.308132 9635 raft_consensus.cc:482] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:28.308369 9635 raft_consensus.cc:504] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:28.310011 9635 leader_election.cc:288] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1974404eef64b4eaef213561a437203 (127.7.209.66:38235), 21a13d6250e14739882c2dfea62b75b5 (127.7.209.67:40063)
I20260929 03:05:28.315011 9627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.310518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:58455 (local address 127.7.209.67:40063)
0929 03:05:28.310694 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:28.310700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:28.310936 (+ 236us) server_negotiation.cc:407] Connection header received
0929 03:05:28.311096 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:28.311100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:28.311166 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:28.311251 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:28.311636 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.312487 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.313923 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.314338 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.314460 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.314531 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:28.314777 (+ 246us) server_negotiation.cc:299] Negotiation successful
0929 03:05:28.314864 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 03:05:28.315071 9625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.310491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:60641 (local address 127.7.209.66:38235)
0929 03:05:28.310693 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:28.310699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:28.311101 (+ 402us) server_negotiation.cc:407] Connection header received
0929 03:05:28.311226 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:28.311230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:28.311296 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:28.311388 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:28.312067 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.312807 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.314056 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.314497 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.314607 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.314678 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:28.314863 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 03:05:28.314955 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260929 03:05:28.315119 9659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.310433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:38235 (local address 127.7.209.65:60641)
0929 03:05:28.310940 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:28.310956 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:28.311116 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:28.311454 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:28.311458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:28.311507 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:28.311893 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:28.311904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.312914 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.312918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:28.313922 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:28.313932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.314606 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.314610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:28.314748 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.314766 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 03:05:28.314834 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 03:05:28.314904 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":4,"thread_start_us":206,"threads_started":1}
I20260929 03:05:28.315371 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.310423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:40063 (local address 127.7.209.65:58455)
0929 03:05:28.310809 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:28.310822 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:28.310962 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:28.311271 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:28.311275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:28.311292 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:28.311495 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:28.311505 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.312620 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.312624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:28.313805 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:28.313815 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.314499 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.314504 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:28.314655 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.314677 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 03:05:28.314747 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 03:05:28.314830 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":106,"threads_started":1}
I20260929 03:05:28.315861 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3cc6e14336049e19d4807ce59506ad6" candidate_uuid: "e70590ac886b4d859494447ab01251ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1974404eef64b4eaef213561a437203" is_pre_election: true
I20260929 03:05:28.316172 9417 raft_consensus.cc:2454] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e70590ac886b4d859494447ab01251ff in term 0.
I20260929 03:05:28.316653 9558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3cc6e14336049e19d4807ce59506ad6" candidate_uuid: "e70590ac886b4d859494447ab01251ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a13d6250e14739882c2dfea62b75b5" is_pre_election: true
I20260929 03:05:28.316681 9208 leader_election.cc:302] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1974404eef64b4eaef213561a437203, e70590ac886b4d859494447ab01251ff; no voters:
I20260929 03:05:28.316798 9558 raft_consensus.cc:2454] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e70590ac886b4d859494447ab01251ff in term 0.
I20260929 03:05:28.317032 9635 raft_consensus.cc:2781] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:28.317096 9635 raft_consensus.cc:482] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:28.317145 9635 raft_consensus.cc:3037] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:28.318781 9635 raft_consensus.cc:504] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:28.319128 9635 leader_election.cc:288] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [CANDIDATE]: Term 1 election: Requested vote from peers c1974404eef64b4eaef213561a437203 (127.7.209.66:38235), 21a13d6250e14739882c2dfea62b75b5 (127.7.209.67:40063)
I20260929 03:05:28.319482 9417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3cc6e14336049e19d4807ce59506ad6" candidate_uuid: "e70590ac886b4d859494447ab01251ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1974404eef64b4eaef213561a437203"
I20260929 03:05:28.319482 9558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3cc6e14336049e19d4807ce59506ad6" candidate_uuid: "e70590ac886b4d859494447ab01251ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a13d6250e14739882c2dfea62b75b5"
I20260929 03:05:28.319615 9417 raft_consensus.cc:3037] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:28.319615 9558 raft_consensus.cc:3037] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:28.321091 9417 raft_consensus.cc:2454] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e70590ac886b4d859494447ab01251ff in term 1.
I20260929 03:05:28.321120 9558 raft_consensus.cc:2454] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e70590ac886b4d859494447ab01251ff in term 1.
I20260929 03:05:28.321403 9209 leader_election.cc:302] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a13d6250e14739882c2dfea62b75b5, e70590ac886b4d859494447ab01251ff; no voters:
I20260929 03:05:28.321568 9635 raft_consensus.cc:2781] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:28.321815 9635 raft_consensus.cc:686] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 1 LEADER]: Becoming Leader. State: Replica: e70590ac886b4d859494447ab01251ff, State: Running, Role: LEADER
I20260929 03:05:28.322124 9635 consensus_queue.cc:237] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:28.324915 9118 catalog_manager.cc:5495] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff reported cstate change: term changed from 0 to 1, leader changed from <none> to e70590ac886b4d859494447ab01251ff (127.7.209.65). New cstate: current_term: 1 leader_uuid: "e70590ac886b4d859494447ab01251ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } health_report { overall_health: UNKNOWN } } }
Sep 29 03:05:28 dist-test-slave-bdps krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651126, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:28.400906 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.392025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47552 (local address 127.7.209.126:36121)
0929 03:05:28.392318 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:28.392323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:28.392479 (+ 156us) server_negotiation.cc:407] Connection header received
0929 03:05:28.392632 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:28.392635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:28.392683 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:28.392774 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:28.393726 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.394319 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.395121 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.395350 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.395437 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.397994 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:28.397997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:28.397999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:28.398021 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:28.399272 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:28.399710 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:28.399712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:28.399714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:28.399790 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:28.400074 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:28.400076 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:28.400077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:28.400252 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:28.400323 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:28.400609 (+ 286us) server_negotiation.cc:299] Negotiation successful
0929 03:05:28.400713 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":199,"thread_start_us":98,"threads_started":1}
I20260929 03:05:28.400960 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.391994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:36121 (local address 127.0.0.1:47552)
0929 03:05:28.392379 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:28.392392 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:28.392542 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:28.392810 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:28.392813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:28.393406 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:28.393559 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:28.393567 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.394422 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.394426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:28.395011 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:28.395019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.395446 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.395449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:28.395575 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.396206 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:28.396220 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:28.397873 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:28.399368 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:28.399376 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:28.399382 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:28.399637 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:28.399889 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:28.399892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:28.399894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:28.399986 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:28.400336 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:28.400343 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:28.400464 (+ 121us) client_negotiation.cc:769] Sending connection context
0929 03:05:28.400564 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 03:05:28.400790 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":34,"thread_start_us":119,"threads_started":1}
I20260929 03:05:28.405378 9118 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:47552:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 03:05:28.406268 9118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:28.411515 9397 tablet_service.cc:1467] Processing CreateTablet for tablet c277ba3a62a449c3a9174d9dada04914 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=09d9f264b9f44ce08d4332fd196908f5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 03:05:28.411509 9538 tablet_service.cc:1467] Processing CreateTablet for tablet c277ba3a62a449c3a9174d9dada04914 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=09d9f264b9f44ce08d4332fd196908f5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 03:05:28.411509 9257 tablet_service.cc:1467] Processing CreateTablet for tablet c277ba3a62a449c3a9174d9dada04914 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=09d9f264b9f44ce08d4332fd196908f5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 03:05:28.411957 9397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c277ba3a62a449c3a9174d9dada04914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:28.411957 9538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c277ba3a62a449c3a9174d9dada04914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:28.411962 9257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c277ba3a62a449c3a9174d9dada04914. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:28.414878 9628 tablet_bootstrap.cc:492] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: Bootstrap starting.
I20260929 03:05:28.415465 9629 tablet_bootstrap.cc:492] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff: Bootstrap starting.
I20260929 03:05:28.415464 9630 tablet_bootstrap.cc:492] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5: Bootstrap starting.
I20260929 03:05:28.415895 9628 tablet_bootstrap.cc:654] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:28.416949 9630 tablet_bootstrap.cc:654] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:28.417038 9629 tablet_bootstrap.cc:654] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:28.418396 9628 tablet_bootstrap.cc:492] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: No bootstrap required, opened a new log
I20260929 03:05:28.418499 9628 ts_tablet_manager.cc:1397] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260929 03:05:28.418839 9629 tablet_bootstrap.cc:492] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff: No bootstrap required, opened a new log
I20260929 03:05:28.418933 9629 ts_tablet_manager.cc:1397] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260929 03:05:28.418973 9628 raft_consensus.cc:348] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:28.419091 9628 raft_consensus.cc:374] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:28.419114 9628 raft_consensus.cc:729] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1974404eef64b4eaef213561a437203, State: Initialized, Role: FOLLOWER
I20260929 03:05:28.419231 9628 consensus_queue.cc:260] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:28.419275 9630 tablet_bootstrap.cc:492] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5: No bootstrap required, opened a new log
I20260929 03:05:28.419353 9630 ts_tablet_manager.cc:1397] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260929 03:05:28.419516 9628 ts_tablet_manager.cc:1428] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260929 03:05:28.419602 9629 raft_consensus.cc:348] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:28.419724 9629 raft_consensus.cc:374] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:28.419761 9629 raft_consensus.cc:729] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70590ac886b4d859494447ab01251ff, State: Initialized, Role: FOLLOWER
I20260929 03:05:28.419709 9636 tablet_replica.cc:439] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } } }
I20260929 03:05:28.419850 9636 tablet_replica.cc:442] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:28.419993 9630 raft_consensus.cc:348] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:28.420110 9630 raft_consensus.cc:374] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:28.420185 9630 raft_consensus.cc:729] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a13d6250e14739882c2dfea62b75b5, State: Initialized, Role: FOLLOWER
I20260929 03:05:28.420343 9630 consensus_queue.cc:260] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:28.420639 9634 tablet_replica.cc:439] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } } }
I20260929 03:05:28.420845 9634 tablet_replica.cc:442] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:28.420111 9629 consensus_queue.cc:260] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:28.421058 9630 ts_tablet_manager.cc:1428] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 03:05:28.422240 9635 tablet_replica.cc:439] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } } }
I20260929 03:05:28.422380 9635 tablet_replica.cc:442] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:28.422974 9629 ts_tablet_manager.cc:1428] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260929 03:05:28.630074 9636 raft_consensus.cc:482] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:28.630293 9636 raft_consensus.cc:504] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:28.630857 9636 leader_election.cc:288] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e70590ac886b4d859494447ab01251ff (127.7.209.65:38627), 21a13d6250e14739882c2dfea62b75b5 (127.7.209.67:40063)
I20260929 03:05:28.631469 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c277ba3a62a449c3a9174d9dada04914" candidate_uuid: "c1974404eef64b4eaef213561a437203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70590ac886b4d859494447ab01251ff" is_pre_election: true
I20260929 03:05:28.631469 9558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c277ba3a62a449c3a9174d9dada04914" candidate_uuid: "c1974404eef64b4eaef213561a437203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a13d6250e14739882c2dfea62b75b5" is_pre_election: true
I20260929 03:05:28.631665 9558 raft_consensus.cc:2454] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1974404eef64b4eaef213561a437203 in term 0.
I20260929 03:05:28.631661 9277 raft_consensus.cc:2454] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1974404eef64b4eaef213561a437203 in term 0.
I20260929 03:05:28.632002 9349 leader_election.cc:302] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a13d6250e14739882c2dfea62b75b5, c1974404eef64b4eaef213561a437203; no voters:
I20260929 03:05:28.632220 9636 raft_consensus.cc:2781] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:28.632279 9636 raft_consensus.cc:482] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:28.632311 9636 raft_consensus.cc:3037] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:28.633555 9636 raft_consensus.cc:504] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:28.633921 9636 leader_election.cc:288] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [CANDIDATE]: Term 1 election: Requested vote from peers e70590ac886b4d859494447ab01251ff (127.7.209.65:38627), 21a13d6250e14739882c2dfea62b75b5 (127.7.209.67:40063)
I20260929 03:05:28.634325 9277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c277ba3a62a449c3a9174d9dada04914" candidate_uuid: "c1974404eef64b4eaef213561a437203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70590ac886b4d859494447ab01251ff"
I20260929 03:05:28.634326 9558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c277ba3a62a449c3a9174d9dada04914" candidate_uuid: "c1974404eef64b4eaef213561a437203" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a13d6250e14739882c2dfea62b75b5"
I20260929 03:05:28.634440 9277 raft_consensus.cc:3037] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:28.634443 9558 raft_consensus.cc:3037] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:28.635492 9277 raft_consensus.cc:2454] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1974404eef64b4eaef213561a437203 in term 1.
I20260929 03:05:28.635520 9558 raft_consensus.cc:2454] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1974404eef64b4eaef213561a437203 in term 1.
I20260929 03:05:28.635797 9351 leader_election.cc:302] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1974404eef64b4eaef213561a437203, e70590ac886b4d859494447ab01251ff; no voters:
I20260929 03:05:28.635968 9636 raft_consensus.cc:2781] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:28.636168 9636 raft_consensus.cc:686] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [term 1 LEADER]: Becoming Leader. State: Replica: c1974404eef64b4eaef213561a437203, State: Running, Role: LEADER
I20260929 03:05:28.636320 9636 consensus_queue.cc:237] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } }
I20260929 03:05:28.637009 9672 tablet_replica.cc:439] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: TxnStatusTablet state changed. Reason: New leader c1974404eef64b4eaef213561a437203. Latest consensus state: current_term: 1 leader_uuid: "c1974404eef64b4eaef213561a437203" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } } }
I20260929 03:05:28.637120 9672 tablet_replica.cc:442] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: This TxnStatusTablet replica's current role is: LEADER
I20260929 03:05:28.637370 9674 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 03:05:28.637550 9674 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 03:05:28.638310 9120 catalog_manager.cc:5495] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1974404eef64b4eaef213561a437203 (127.7.209.66). New cstate: current_term: 1 leader_uuid: "c1974404eef64b4eaef213561a437203" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } health_report { overall_health: UNKNOWN } } }
I20260929 03:05:28.660126 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.656446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:38235 (local address 127.0.0.1:55696)
0929 03:05:28.656627 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:28.656645 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:28.656770 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:28.657067 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:28.657070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:28.657078 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 03:05:28.657207 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:28.657213 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.658220 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.658223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:28.658886 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:28.658895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.659342 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.659345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:28.659444 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.659494 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 03:05:28.659802 (+ 308us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 03:05:28.659820 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 03:05:28.659879 (+ 59us) client_negotiation.cc:240] Negotiation successful
0929 03:05:28.659990 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 03:05:28.660126 9625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.656548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55696 (local address 127.7.209.66:38235)
0929 03:05:28.656691 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:28.656695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:28.656711 (+ 16us) server_negotiation.cc:407] Connection header received
0929 03:05:28.656912 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:28.656915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:28.656968 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:28.657034 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:28.657311 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.658105 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.659004 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.659239 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.659331 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.659627 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:28.659721 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:28.659782 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:28.659899 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 03:05:28.659961 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:05:28.663306 9558 raft_consensus.cc:1261] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5 [term 1 FOLLOWER]: Refusing update from remote peer c1974404eef64b4eaef213561a437203: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:28.663491 9277 raft_consensus.cc:1261] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff [term 1 FOLLOWER]: Refusing update from remote peer c1974404eef64b4eaef213561a437203: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:28.663914 9636 consensus_queue.cc:1035] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:28.664224 9636 consensus_queue.cc:1035] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:28.665490 9634 tablet_replica.cc:439] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5: TxnStatusTablet state changed. Reason: New leader c1974404eef64b4eaef213561a437203. Latest consensus state: current_term: 1 leader_uuid: "c1974404eef64b4eaef213561a437203" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } } }
I20260929 03:05:28.665657 9634 tablet_replica.cc:442] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:28.665729 9635 tablet_replica.cc:439] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff: TxnStatusTablet state changed. Reason: New leader c1974404eef64b4eaef213561a437203. Latest consensus state: current_term: 1 leader_uuid: "c1974404eef64b4eaef213561a437203" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } } }
I20260929 03:05:28.665858 9635 tablet_replica.cc:442] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:28.668011 9636 tablet_replica.cc:439] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1974404eef64b4eaef213561a437203" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } } }
I20260929 03:05:28.668169 9636 tablet_replica.cc:442] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: This TxnStatusTablet replica's current role is: LEADER
I20260929 03:05:28.668447 9636 tablet_replica.cc:439] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c1974404eef64b4eaef213561a437203" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } } }
I20260929 03:05:28.668532 9636 tablet_replica.cc:442] T c277ba3a62a449c3a9174d9dada04914 P c1974404eef64b4eaef213561a437203: This TxnStatusTablet replica's current role is: LEADER
I20260929 03:05:28.668608 9677 mvcc.cc:205] Tried to move back new op lower bound from 7334507023000051712 to 7334507022895828992. Current Snapshot: MvccSnapshot[applied={T|T < 7334507023000051712}]
I20260929 03:05:28.669371 9634 tablet_replica.cc:439] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1974404eef64b4eaef213561a437203" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } } }
I20260929 03:05:28.669512 9634 tablet_replica.cc:442] T c277ba3a62a449c3a9174d9dada04914 P 21a13d6250e14739882c2dfea62b75b5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:28.672158 9635 tablet_replica.cc:439] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c1974404eef64b4eaef213561a437203" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e70590ac886b4d859494447ab01251ff" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38627 } } peers { permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 } } peers { permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 } } }
I20260929 03:05:28.672297 9635 tablet_replica.cc:442] T c277ba3a62a449c3a9174d9dada04914 P e70590ac886b4d859494447ab01251ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:28.681643 9687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.675725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52746 (local address 127.7.209.65:38627)
0929 03:05:28.675939 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:28.675943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:28.675957 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:05:28.676053 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:28.676056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:28.676135 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:28.676283 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:28.676593 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.677711 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.679035 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.679460 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.679559 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.679949 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:28.680700 (+ 751us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:28.680820 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:28.681282 (+ 462us) server_negotiation.cc:299] Negotiation successful
0929 03:05:28.681351 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260929 03:05:28.695019 9625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.691230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:55201 (local address 127.7.209.66:38235)
0929 03:05:28.691376 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:28.691381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:28.691612 (+ 231us) server_negotiation.cc:407] Connection header received
0929 03:05:28.691738 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:28.691741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:28.691791 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:28.691852 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:28.692168 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.692682 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.694014 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.694229 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.694302 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.694630 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:28.694711 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:28.694757 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:28.694852 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 03:05:28.694903 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 03:05:28.695137 9690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.691118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:38235 (local address 127.7.209.65:55201)
0929 03:05:28.691526 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:28.691542 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:28.691655 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:28.691870 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:28.691873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:28.691880 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 03:05:28.692071 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:28.692078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.692770 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.692774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:28.693916 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:28.693927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.694316 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.694320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:28.694455 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.694508 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 03:05:28.694775 (+ 267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 03:05:28.694780 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 03:05:28.694841 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 03:05:28.694926 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":134,"threads_started":1}
I20260929 03:05:28.702495 9558 raft_consensus.cc:1261] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5 [term 1 FOLLOWER]: Refusing update from remote peer e70590ac886b4d859494447ab01251ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:28.702749 9417 raft_consensus.cc:1261] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203 [term 1 FOLLOWER]: Refusing update from remote peer e70590ac886b4d859494447ab01251ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:28.703044 9635 consensus_queue.cc:1035] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a13d6250e14739882c2dfea62b75b5" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 40063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:28.703369 9660 consensus_queue.cc:1035] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1974404eef64b4eaef213561a437203" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:28.733460 9687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.724417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:38667 (local address 127.7.209.65:38627)
0929 03:05:28.724548 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:28.724552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:28.729989 (+ 5437us) server_negotiation.cc:407] Connection header received
0929 03:05:28.730120 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:28.730123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:28.730181 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:28.730243 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:28.730516 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.731077 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.732374 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.732633 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.732721 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.733005 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:28.733101 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:28.733158 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:28.733277 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 03:05:28.733336 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:05:28.733502 9696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.724310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:38627 (local address 127.7.209.66:38667)
0929 03:05:28.729894 (+ 5584us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:28.729911 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:28.730033 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:28.730262 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:28.730265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:28.730273 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 03:05:28.730397 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:28.730403 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.731176 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.731179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:28.732228 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:28.732239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.732727 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.732731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:28.732822 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.732879 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 03:05:28.733170 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 03:05:28.733176 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 03:05:28.733233 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 03:05:28.733323 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5426,"thread_start_us":151,"threads_started":1}
I20260929 03:05:28.764786 9118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:28.764989 9118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:28.767421 9118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 772b38eb9f0e4a93969314ef309837d4: Sending DeleteTablet for 3 replicas of tablet f3cc6e14336049e19d4807ce59506ad6
I20260929 03:05:28.768009 9397 tablet_service.cc:1514] Processing DeleteTablet for tablet f3cc6e14336049e19d4807ce59506ad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:28 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:55690
I20260929 03:05:28.768074 9257 tablet_service.cc:1514] Processing DeleteTablet for tablet f3cc6e14336049e19d4807ce59506ad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:28 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:52740
I20260929 03:05:28.768671 9700 tablet_replica.cc:330] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203: stopping tablet replica
I20260929 03:05:28.768890 9700 raft_consensus.cc:2229] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:28.769101 9700 raft_consensus.cc:2258] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:28.769213 9538 tablet_service.cc:1514] Processing DeleteTablet for tablet f3cc6e14336049e19d4807ce59506ad6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:28 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:40808
I20260929 03:05:28.770124 9700 ts_tablet_manager.cc:1905] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:28.771368 9699 tablet_replica.cc:330] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff: stopping tablet replica
I20260929 03:05:28.771581 9699 raft_consensus.cc:2229] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:05:28.771993 9699 raft_consensus.cc:2258] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:28.772763 9699 ts_tablet_manager.cc:1905] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:28.773800 9700 ts_tablet_manager.cc:1918] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:05:28.774524 9700 log.cc:1196] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/f3cc6e14336049e19d4807ce59506ad6
I20260929 03:05:28.775350 9700 ts_tablet_manager.cc:1939] T f3cc6e14336049e19d4807ce59506ad6 P c1974404eef64b4eaef213561a437203: Deleting consensus metadata
I20260929 03:05:28.775663 9701 tablet_replica.cc:330] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5: stopping tablet replica
I20260929 03:05:28.776464 9701 raft_consensus.cc:2229] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:28.776747 9701 raft_consensus.cc:2258] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:28.776750 9699 ts_tablet_manager.cc:1918] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:05:28.776824 9699 log.cc:1196] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-0/wal/wals/f3cc6e14336049e19d4807ce59506ad6
I20260929 03:05:28.777156 9699 ts_tablet_manager.cc:1939] T f3cc6e14336049e19d4807ce59506ad6 P e70590ac886b4d859494447ab01251ff: Deleting consensus metadata
I20260929 03:05:28.777238 9104 catalog_manager.cc:4841] TS c1974404eef64b4eaef213561a437203 (127.7.209.66:38235): tablet f3cc6e14336049e19d4807ce59506ad6 (table test-table [id=f3ac5343afb149e1a6519e0c85a44466]) successfully deleted
I20260929 03:05:28.777966 9107 catalog_manager.cc:4841] TS e70590ac886b4d859494447ab01251ff (127.7.209.65:38627): tablet f3cc6e14336049e19d4807ce59506ad6 (table test-table [id=f3ac5343afb149e1a6519e0c85a44466]) successfully deleted
I20260929 03:05:28.778173 9701 ts_tablet_manager.cc:1905] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:28.782061 9701 ts_tablet_manager.cc:1918] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:05:28.782217 9701 log.cc:1196] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/f3cc6e14336049e19d4807ce59506ad6
I20260929 03:05:28.782558 9701 ts_tablet_manager.cc:1939] T f3cc6e14336049e19d4807ce59506ad6 P 21a13d6250e14739882c2dfea62b75b5: Deleting consensus metadata
I20260929 03:05:28.783468 9105 catalog_manager.cc:4841] TS 21a13d6250e14739882c2dfea62b75b5 (127.7.209.67:40063): tablet f3cc6e14336049e19d4807ce59506ad6 (table test-table [id=f3ac5343afb149e1a6519e0c85a44466]) successfully deleted
Sep 29 03:05:28 dist-test-slave-bdps krb5kdc[9075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651127, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
I20260929 03:05:28.791251 9687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.774334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52752 (local address 127.7.209.65:38627)
0929 03:05:28.774462 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:28.774467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:28.778711 (+ 4244us) server_negotiation.cc:407] Connection header received
0929 03:05:28.778947 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:28.778950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:28.779009 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:28.779080 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:28.780156 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.780929 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.782270 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.782630 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.782716 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.786926 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:28.786940 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:28.786959 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:28.786989 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:28.789025 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:28.789523 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:28.789530 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:28.789532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:28.789676 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:28.789975 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:28.789978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:28.789980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:28.790371 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:28.790675 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:28.790836 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 03:05:28.790985 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260929 03:05:28.792627 9317 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52752
I20260929 03:05:28.802843 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:28.794877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47560 (local address 127.7.209.126:36121)
0929 03:05:28.795053 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:28.795058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:28.795134 (+ 76us) server_negotiation.cc:407] Connection header received
0929 03:05:28.795251 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:28.795254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:28.795309 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:28.795376 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:28.796062 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.796651 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.797451 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:28.797770 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:28.797859 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:28.799394 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:28.799399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:28.799402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:28.799430 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:28.800858 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:28.801266 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:28.801269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:28.801271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:28.801355 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:28.801676 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:28.801679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:28.801681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:28.801910 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:28.802076 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:28.802375 (+ 299us) server_negotiation.cc:299] Negotiation successful
0929 03:05:28.802511 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260929 03:05:28.803354 9118 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47560
I20260929 03:05:28.804973 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 9192
I20260929 03:05:28.813868 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 9332
I20260929 03:05:28.820679 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 9472
I20260929 03:05:28.825490 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 9088
2026-09-29T03:05:28Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3440 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:05:28 dist-test-slave-bdps krb5kdc[9708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:05:28 dist-test-slave-bdps krb5kdc[9708](info): set up 2 sockets
Sep 29 03:05:28 dist-test-slave-bdps krb5kdc[9708](info): commencing operation
krb5kdc: starting...
W20260929 03:05:29.781731 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:05:29 dist-test-slave-bdps krb5kdc[9708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:05:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:05:29.912892 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:39725
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:39725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:30.041468 9721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:30.041842 9721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:30.041909 9721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:30.045904 9721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:30.045956 9721 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 03:05:30.045979 9721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:30.046005 9721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:30.046036 9721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:30.046064 9721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:30.051488 9721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:39725
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:39725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:30.052740 9721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:30.054052 9721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:30.063773 9726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:30.063855 9721 server_base.cc:1034] running on GCE node
W20260929 03:05:30.063825 9727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:30.063773 9729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:30.064810 9721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:30.066277 9721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:30.067441 9721 hybrid_clock.cc:648] HybridClock initialized: now 1790651130067421 us; error 35 us; skew 500 ppm
Sep 29 03:05:30 dist-test-slave-bdps krb5kdc[9708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651130, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:30.070863 9721 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:05:30.072183 9721 webserver.cc:460] Webserver started at http://127.7.209.126:40195/ using document root <none> and password file <none>
I20260929 03:05:30.072894 9721 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:30.072965 9721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:30.073195 9721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:30.075800 9721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "1c4cf66131cb462a92ff2c423e14c1ea"
format_stamp: "Formatted at 2026-09-29 03:05:30 on dist-test-slave-bdps"
server_key: "bdc7385323ff8701d023c002cdb43db0"
server_key_iv: "03f3a61a3cbf83464c9e5151f926ba06"
server_key_version: "encryptionkey@0"
I20260929 03:05:30.076412 9721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "1c4cf66131cb462a92ff2c423e14c1ea"
format_stamp: "Formatted at 2026-09-29 03:05:30 on dist-test-slave-bdps"
server_key: "bdc7385323ff8701d023c002cdb43db0"
server_key_iv: "03f3a61a3cbf83464c9e5151f926ba06"
server_key_version: "encryptionkey@0"
I20260929 03:05:30.081571 9721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 03:05:30.084822 9736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:30.086467 9721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:30.086591 9721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "1c4cf66131cb462a92ff2c423e14c1ea"
format_stamp: "Formatted at 2026-09-29 03:05:30 on dist-test-slave-bdps"
server_key: "bdc7385323ff8701d023c002cdb43db0"
server_key_iv: "03f3a61a3cbf83464c9e5151f926ba06"
server_key_version: "encryptionkey@0"
I20260929 03:05:30.086721 9721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:30.104472 9721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:30.107934 9721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:30.108160 9721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:30.117559 9721 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:39725
I20260929 03:05:30.117589 9798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:39725 every 8 connection(s)
I20260929 03:05:30.118752 9721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:30.121059 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 9721
I20260929 03:05:30.121187 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:30.121484 8005 external_mini_cluster.cc:1428] Setting key 97ed127909d5ad2bfa09ea28e79e179a
I20260929 03:05:30.122498 9799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:30.129595 9799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea: Bootstrap starting.
Sep 29 03:05:30 dist-test-slave-bdps krb5kdc[9708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:30.132567 9799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:30.133926 9799 log.cc:824] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:30.136252 9799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea: No bootstrap required, opened a new log
I20260929 03:05:30.137054 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.123207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33676 (local address 127.7.209.126:39725)
0929 03:05:30.123656 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:30.123674 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:30.123715 (+ 41us) server_negotiation.cc:407] Connection header received
0929 03:05:30.124442 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:30.124460 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:30.124846 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:30.125184 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:30.126149 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.127350 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.128614 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.129115 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.129241 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.132032 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:30.132039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:30.132056 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:30.132088 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:30.134037 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.134462 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.134467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.134472 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.134598 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.134864 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.134867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.134868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.135354 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:30.135547 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:30.135737 (+ 190us) server_negotiation.cc:299] Negotiation successful
0929 03:05:30.136037 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":135,"threads_started":1}
I20260929 03:05:30.139878 9799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4cf66131cb462a92ff2c423e14c1ea" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39725 } }
I20260929 03:05:30.140105 9799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:30.140141 9799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c4cf66131cb462a92ff2c423e14c1ea, State: Initialized, Role: FOLLOWER
I20260929 03:05:30.140517 9799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4cf66131cb462a92ff2c423e14c1ea" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39725 } }
I20260929 03:05:30.140630 9799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:30.140674 9799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:30.140745 9799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:30.141501 9799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4cf66131cb462a92ff2c423e14c1ea" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39725 } }
I20260929 03:05:30.141850 9799 leader_election.cc:302] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c4cf66131cb462a92ff2c423e14c1ea; no voters:
I20260929 03:05:30.142110 9799 leader_election.cc:288] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:30.142210 9804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:30.142429 9804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [term 1 LEADER]: Becoming Leader. State: Replica: 1c4cf66131cb462a92ff2c423e14c1ea, State: Running, Role: LEADER
I20260929 03:05:30.142758 9804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4cf66131cb462a92ff2c423e14c1ea" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39725 } }
I20260929 03:05:30.143366 9799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:30.145239 9805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c4cf66131cb462a92ff2c423e14c1ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4cf66131cb462a92ff2c423e14c1ea" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39725 } } }
I20260929 03:05:30.145426 9805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:30.146311 9806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c4cf66131cb462a92ff2c423e14c1ea. Latest consensus state: current_term: 1 leader_uuid: "1c4cf66131cb462a92ff2c423e14c1ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c4cf66131cb462a92ff2c423e14c1ea" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39725 } } }
I20260929 03:05:30.146449 9806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea [sys.catalog]: This master's current role is: LEADER
W20260929 03:05:30.148763 9818 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:05:30.148830 9818 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:05:30.148914 9819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:30.149847 9819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:30.157308 9819 catalog_manager.cc:1347] Generated new cluster ID: cfbe791638d24ebcbe3350a1e0c12e6f
I20260929 03:05:30.157409 9819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:30.177016 9819 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:30.178223 9819 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:30.185036 9819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea: Generated new TSK 0
I20260929 03:05:30.185963 9819 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:05:30.222597 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39725
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:30.368492 9825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:30.368829 9825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:30.368885 9825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:30.373108 9825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:30.373181 9825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:30.373232 9825 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 03:05:30.373261 9825 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 03:05:30.373313 9825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:30.378931 9825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.209.126:39725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:30.380378 9825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:30.381739 9825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:05:30.394371 9825 server_base.cc:1034] running on GCE node
W20260929 03:05:30.394290 9833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:30.394331 9831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:30.394301 9830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:30.395087 9825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:30.395939 9825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:30.397127 9825 hybrid_clock.cc:648] HybridClock initialized: now 1790651130397100 us; error 49 us; skew 500 ppm
Sep 29 03:05:30 dist-test-slave-bdps krb5kdc[9708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651130, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:30.400410 9825 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:05:30.401575 9825 webserver.cc:460] Webserver started at http://127.7.209.65:37047/ using document root <none> and password file <none>
I20260929 03:05:30.402308 9825 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:30.402375 9825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:30.402590 9825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:30.405045 9825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "3fedc543ed7248b68b7f4b66b45d55c3"
format_stamp: "Formatted at 2026-09-29 03:05:30 on dist-test-slave-bdps"
server_key: "f3a72e4a2797b8808b2e2c15e156e8b5"
server_key_iv: "061b4d8fefd040816f384c76881514f5"
server_key_version: "encryptionkey@0"
I20260929 03:05:30.405689 9825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "3fedc543ed7248b68b7f4b66b45d55c3"
format_stamp: "Formatted at 2026-09-29 03:05:30 on dist-test-slave-bdps"
server_key: "f3a72e4a2797b8808b2e2c15e156e8b5"
server_key_iv: "061b4d8fefd040816f384c76881514f5"
server_key_version: "encryptionkey@0"
I20260929 03:05:30.410048 9825 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:30.412734 9840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:30.414165 9825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:30.414278 9825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "3fedc543ed7248b68b7f4b66b45d55c3"
format_stamp: "Formatted at 2026-09-29 03:05:30 on dist-test-slave-bdps"
server_key: "f3a72e4a2797b8808b2e2c15e156e8b5"
server_key_iv: "061b4d8fefd040816f384c76881514f5"
server_key_version: "encryptionkey@0"
I20260929 03:05:30.414378 9825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:30.430806 9825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:30.434562 9825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:30.434767 9825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:30.436478 9825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:30.436529 9825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:30.436576 9825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:30.436606 9825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 03:05:30 dist-test-slave-bdps krb5kdc[9708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651130, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:30.448513 9825 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:35229
I20260929 03:05:30.448810 9955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:35229 every 8 connection(s)
I20260929 03:05:30.449503 9825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:30.451088 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 9825
I20260929 03:05:30.451218 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:05:30.451517 8005 external_mini_cluster.cc:1428] Setting key d98d04600dbd92aaa104063fcb7cc29f
I20260929 03:05:30.451560 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.437483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:35071 (local address 127.7.209.126:39725)
0929 03:05:30.437649 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:30.437653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:30.438918 (+ 1265us) server_negotiation.cc:407] Connection header received
0929 03:05:30.440427 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:30.440431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:30.440506 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:30.440619 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:30.442289 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.442973 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.444425 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.444809 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.444914 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.447808 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:30.447812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:30.447814 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:30.447841 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:30.449338 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.449898 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.449900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.449902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.449998 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.450395 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.450398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.450400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.450612 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:30.450703 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:30.451241 (+ 538us) server_negotiation.cc:299] Negotiation successful
0929 03:05:30.451380 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 03:05:30.452533 9849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.438184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39725 (local address 127.7.209.65:35071)
0929 03:05:30.438743 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:30.438793 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:30.440120 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:30.440862 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:30.440872 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:30.441309 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:30.442115 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:30.442132 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.443105 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.443109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:30.444262 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:30.444275 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.444996 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.445000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:30.445231 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.445928 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:30.445945 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:30.447624 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:30.449448 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:30.449455 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:30.449471 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:30.449790 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:30.450119 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:30.450122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:30.450124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:30.450288 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:30.450715 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:30.450722 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:30.450990 (+ 268us) client_negotiation.cc:769] Sending connection context
0929 03:05:30.451163 (+ 173us) client_negotiation.cc:240] Negotiation successful
0929 03:05:30.451486 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":158,"threads_started":1}
I20260929 03:05:30.454131 9956 heartbeater.cc:347] Connected to a master server at 127.7.209.126:39725
I20260929 03:05:30.454324 9956 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:30.454964 9956 heartbeater.cc:511] Master 127.7.209.126:39725 requested a full tablet report, sending...
I20260929 03:05:30.456866 9752 ts_manager.cc:194] Registered new tserver with Master: 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229)
I20260929 03:05:30.458532 9752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:35071
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:05:30.464426 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.456470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:35773 (local address 127.7.209.126:39725)
0929 03:05:30.456586 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:30.456590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:30.456857 (+ 267us) server_negotiation.cc:407] Connection header received
0929 03:05:30.456962 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:30.456965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:30.457013 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:30.457094 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:30.457558 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.458399 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.459559 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.459898 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.459998 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.461556 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:30.461560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:30.461563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:30.461618 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:30.462937 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.463386 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.463388 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.463390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.463454 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.463750 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.463753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.463754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.463924 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:30.464002 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:30.464211 (+ 209us) server_negotiation.cc:299] Negotiation successful
0929 03:05:30.464305 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 03:05:30.464442 9963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.456377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39725 (local address 127.7.209.65:35773)
0929 03:05:30.456751 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:30.456767 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:30.456882 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:30.457124 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:30.457127 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:30.457331 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:30.457448 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:30.457455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.458536 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.458540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:30.459426 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:30.459436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.460125 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.460130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:30.460249 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.460788 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:30.460802 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:30.461265 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:30.463045 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:30.463048 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:30.463051 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:30.463304 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:30.463524 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:30.463527 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:30.463529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:30.463627 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:30.464006 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:30.464009 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:30.464088 (+ 79us) client_negotiation.cc:769] Sending connection context
0929 03:05:30.464176 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 03:05:30.464286 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":98,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:05:30.484122 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39725
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:30.610517 9965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:30.610807 9965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:30.610873 9965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:30.614832 9965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:30.614886 9965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:30.614940 9965 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 03:05:30.614969 9965 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 03:05:30.615021 9965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:30.620183 9965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.209.126:39725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:30.621299 9965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:30.622454 9965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:30.633239 9971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:30.633235 9970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:30.633255 9973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:30.633279 9965 server_base.cc:1034] running on GCE node
I20260929 03:05:30.633996 9965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:30.634717 9965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:30.635874 9965 hybrid_clock.cc:648] HybridClock initialized: now 1790651130635855 us; error 41 us; skew 500 ppm
Sep 29 03:05:30 dist-test-slave-bdps krb5kdc[9708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651130, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:30.638734 9965 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:05:30.639729 9965 webserver.cc:460] Webserver started at http://127.7.209.66:39665/ using document root <none> and password file <none>
I20260929 03:05:30.640348 9965 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:30.640406 9965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:30.640604 9965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:30.642979 9965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "80e1755b229e41219d14e6b9a14858d0"
format_stamp: "Formatted at 2026-09-29 03:05:30 on dist-test-slave-bdps"
server_key: "93c63b2db87311a4444499308b703987"
server_key_iv: "6900b71209efca1d7e0f0e60df276f15"
server_key_version: "encryptionkey@0"
I20260929 03:05:30.643574 9965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "80e1755b229e41219d14e6b9a14858d0"
format_stamp: "Formatted at 2026-09-29 03:05:30 on dist-test-slave-bdps"
server_key: "93c63b2db87311a4444499308b703987"
server_key_iv: "6900b71209efca1d7e0f0e60df276f15"
server_key_version: "encryptionkey@0"
I20260929 03:05:30.648017 9965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:05:30.650681 9980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:30.652182 9965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:30.652284 9965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "80e1755b229e41219d14e6b9a14858d0"
format_stamp: "Formatted at 2026-09-29 03:05:30 on dist-test-slave-bdps"
server_key: "93c63b2db87311a4444499308b703987"
server_key_iv: "6900b71209efca1d7e0f0e60df276f15"
server_key_version: "encryptionkey@0"
I20260929 03:05:30.652390 9965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:30.678920 9965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:30.682576 9965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:30.682802 9965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:30.684715 9965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:30.684804 9965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:30.684864 9965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:30.684890 9965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 03:05:30 dist-test-slave-bdps krb5kdc[9708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651130, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:30.697053 9965 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:40031
I20260929 03:05:30.697095 10095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:40031 every 8 connection(s)
I20260929 03:05:30.698326 9965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:30.700102 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.685747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:59537 (local address 127.7.209.126:39725)
0929 03:05:30.685877 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:30.685881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:30.687257 (+ 1376us) server_negotiation.cc:407] Connection header received
0929 03:05:30.688509 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:30.688513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:30.688589 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:30.688658 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:30.690518 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.691378 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.692767 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.693061 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.693160 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.696120 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:30.696123 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:30.696125 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:30.696156 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:30.697951 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.698453 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.698456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.698457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.698542 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.698916 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.698920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.698921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.699166 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:30.699269 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:30.699777 (+ 508us) server_negotiation.cc:299] Negotiation successful
0929 03:05:30.699912 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 03:05:30.700984 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.686500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39725 (local address 127.7.209.66:59537)
0929 03:05:30.687071 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:30.687121 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:30.688202 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:30.689202 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:30.689212 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:30.689743 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:30.690354 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:30.690370 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.691505 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.691509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:30.692637 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:30.692648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.693255 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.693260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:30.693435 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.694166 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:30.694183 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:30.695942 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:30.698060 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:30.698065 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:30.698079 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:30.698340 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:30.698624 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:30.698627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:30.698629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:30.698802 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:30.699260 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:30.699266 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:30.699529 (+ 263us) client_negotiation.cc:769] Sending connection context
0929 03:05:30.699702 (+ 173us) client_negotiation.cc:240] Negotiation successful
0929 03:05:30.700053 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":167,"threads_started":1}
I20260929 03:05:30.701673 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 9965
I20260929 03:05:30.701833 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:05:30.702168 8005 external_mini_cluster.cc:1428] Setting key b9ec110792593b8e6e6eb31aa15a13ad
I20260929 03:05:30.702414 10096 heartbeater.cc:347] Connected to a master server at 127.7.209.126:39725
I20260929 03:05:30.702566 10096 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:30.703181 10096 heartbeater.cc:511] Master 127.7.209.126:39725 requested a full tablet report, sending...
I20260929 03:05:30.704310 9753 ts_manager.cc:194] Registered new tserver with Master: 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66:40031)
I20260929 03:05:30.705061 9753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:59537
I20260929 03:05:30.712303 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.704892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:33577 (local address 127.7.209.126:39725)
0929 03:05:30.704999 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:30.705002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:30.705231 (+ 229us) server_negotiation.cc:407] Connection header received
0929 03:05:30.705337 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:30.705340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:30.705385 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:30.705460 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:30.705972 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.706706 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.707790 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.708005 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.708098 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.709191 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:30.709195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:30.709198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:30.709221 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:30.710616 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.711026 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.711030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.711032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.711112 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.711366 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.711370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.711372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.711636 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:30.711725 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:30.711980 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 03:05:30.712105 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 03:05:30.712316 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.704816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39725 (local address 127.7.209.66:33577)
0929 03:05:30.705114 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:30.705170 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:30.705267 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:30.705481 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:30.705483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:30.705716 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:30.705862 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:30.705868 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.706842 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.706846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:30.707686 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:30.707696 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.708119 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.708123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:30.708227 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.708722 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:30.708732 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:30.709060 (+ 328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:30.710728 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:30.710731 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:30.710734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:30.710942 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:30.711194 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:30.711197 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:30.711199 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:30.711287 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:30.711727 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:30.711730 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:30.711828 (+ 98us) client_negotiation.cc:769] Sending connection context
0929 03:05:30.711914 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 03:05:30.712087 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:05:30.732245 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39725
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:30.859802 10105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:30.860093 10105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:30.860141 10105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:30.864233 10105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:30.864286 10105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:30.864323 10105 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 03:05:30.864336 10105 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 03:05:30.864367 10105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:30.869511 10105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.209.126:39725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:30.870841 10105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:30.872207 10105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:30.883800 10113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:30.883844 10111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:30.883810 10110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:30.884176 10105 server_base.cc:1034] running on GCE node
I20260929 03:05:30.884641 10105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:30.885371 10105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:30.886538 10105 hybrid_clock.cc:648] HybridClock initialized: now 1790651130886505 us; error 44 us; skew 500 ppm
Sep 29 03:05:30 dist-test-slave-bdps krb5kdc[9708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651130, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:30.889467 10105 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:05:30.890425 10105 webserver.cc:460] Webserver started at http://127.7.209.67:33559/ using document root <none> and password file <none>
I20260929 03:05:30.891085 10105 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:30.891149 10105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:30.891348 10105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:30.893754 10105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "2daa50738ae7444a982e73f0b39c271e"
format_stamp: "Formatted at 2026-09-29 03:05:30 on dist-test-slave-bdps"
server_key: "ddaf1fbb7abd77352092252ce9483c4a"
server_key_iv: "d80c4ecdc20bdfef7b3001274c1f35b3"
server_key_version: "encryptionkey@0"
I20260929 03:05:30.894349 10105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "2daa50738ae7444a982e73f0b39c271e"
format_stamp: "Formatted at 2026-09-29 03:05:30 on dist-test-slave-bdps"
server_key: "ddaf1fbb7abd77352092252ce9483c4a"
server_key_iv: "d80c4ecdc20bdfef7b3001274c1f35b3"
server_key_version: "encryptionkey@0"
I20260929 03:05:30.898542 10105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:30.901091 10120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:30.902493 10105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:05:30.902593 10105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "2daa50738ae7444a982e73f0b39c271e"
format_stamp: "Formatted at 2026-09-29 03:05:30 on dist-test-slave-bdps"
server_key: "ddaf1fbb7abd77352092252ce9483c4a"
server_key_iv: "d80c4ecdc20bdfef7b3001274c1f35b3"
server_key_version: "encryptionkey@0"
I20260929 03:05:30.902700 10105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:30.920351 10105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:30.923923 10105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:30.924113 10105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:30.925766 10105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:30.925819 10105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:30.925879 10105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:30.925906 10105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 03:05:30 dist-test-slave-bdps krb5kdc[9708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651130, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:30.937427 10105 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:33525
I20260929 03:05:30.937494 10235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:33525 every 8 connection(s)
I20260929 03:05:30.938460 10105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:30.938992 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 10105
I20260929 03:05:30.939132 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:05:30.939517 8005 external_mini_cluster.cc:1428] Setting key f785359150975d1f0ab80f06c3621660
I20260929 03:05:30.941267 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.926759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:49167 (local address 127.7.209.126:39725)
0929 03:05:30.926887 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:30.926891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:30.928151 (+ 1260us) server_negotiation.cc:407] Connection header received
0929 03:05:30.929420 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:30.929424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:30.929496 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:30.929594 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:30.931355 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.932221 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.934054 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.934284 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.934370 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.937333 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:30.937336 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:30.937339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:30.937367 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:30.938862 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.939673 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.939677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.939679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.939787 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.940106 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.940109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.940111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.940344 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:30.940448 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:30.940991 (+ 543us) server_negotiation.cc:299] Negotiation successful
0929 03:05:30.941124 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:05:30.942270 10129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.927390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39725 (local address 127.7.209.67:49167)
0929 03:05:30.927986 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:30.928032 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:30.929132 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:30.929859 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:30.929870 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:30.930361 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:30.931177 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:30.931200 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.932357 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.932362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:30.933927 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:30.933939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.934446 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.934451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:30.934665 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.935261 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:30.935283 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:30.937152 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:30.939159 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:30.939165 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:30.939180 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:30.939507 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:30.939890 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:30.939893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:30.939894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:30.940030 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:30.940447 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:30.940454 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:30.940733 (+ 279us) client_negotiation.cc:769] Sending connection context
0929 03:05:30.940934 (+ 201us) client_negotiation.cc:240] Negotiation successful
0929 03:05:30.941265 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":213,"threads_started":1}
I20260929 03:05:30.943466 10236 heartbeater.cc:347] Connected to a master server at 127.7.209.126:39725
I20260929 03:05:30.943614 10236 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:30.944151 10236 heartbeater.cc:511] Master 127.7.209.126:39725 requested a full tablet report, sending...
I20260929 03:05:30.945214 9752 ts_manager.cc:194] Registered new tserver with Master: 2daa50738ae7444a982e73f0b39c271e (127.7.209.67:33525)
I20260929 03:05:30.946031 9752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:49167
I20260929 03:05:30.952196 10242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.945715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39725 (local address 127.7.209.67:36013)
0929 03:05:30.945969 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:30.945981 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:30.946076 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:30.946291 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:30.946294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:30.946479 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:30.946575 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:30.946580 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.947313 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.947316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:30.947936 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:30.947944 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.948369 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.948373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:30.948476 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.948965 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:30.948976 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:30.949388 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:30.950870 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:30.950873 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:30.950875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:30.951071 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:30.951356 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:30.951359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:30.951361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:30.951447 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:30.951754 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:30.951756 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:30.951837 (+ 81us) client_negotiation.cc:769] Sending connection context
0929 03:05:30.951912 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 03:05:30.952031 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":101,"threads_started":1}
I20260929 03:05:30.952250 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.945746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:36013 (local address 127.7.209.126:39725)
0929 03:05:30.945856 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:30.945860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:30.946048 (+ 188us) server_negotiation.cc:407] Connection header received
0929 03:05:30.946148 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:30.946152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:30.946196 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:30.946268 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:30.946675 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.947190 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.948040 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.948272 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.948356 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.949503 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:30.949507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:30.949511 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:30.949536 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:30.950773 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.951181 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.951184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.951185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.951272 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.951512 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.951514 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.951516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.951686 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:30.951762 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:30.951960 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 03:05:30.952082 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 03:05:30.953899 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 03:05:30 dist-test-slave-bdps krb5kdc[9708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 03:05:30 dist-test-slave-bdps krb5kdc[9708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:30.981805 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.973391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33688 (local address 127.7.209.126:39725)
0929 03:05:30.973525 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:30.973529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:30.973541 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:05:30.973592 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:30.973595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:30.973676 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:30.973753 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:30.974546 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.975121 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.975993 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:30.976232 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:30.976326 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:30.978925 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:30.978927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:30.978930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:30.978960 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:30.980148 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.980555 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.980558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.980560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.980647 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:30.980903 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:30.980906 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:30.980908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:30.981106 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:30.981255 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:30.981460 (+ 205us) server_negotiation.cc:299] Negotiation successful
0929 03:05:30.981589 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 03:05:30.985139 9752 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:05:30.989082 9752 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:31.005782 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.000683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51778 (local address 127.7.209.67:33525)
0929 03:05:31.001193 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.001206 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.001232 (+ 26us) server_negotiation.cc:407] Connection header received
0929 03:05:31.001351 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.001357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.001524 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.001715 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:31.001896 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.002768 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.004460 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.005049 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.005162 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.005365 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.005462 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.005573 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":141,"threads_started":1}
I20260929 03:05:31.005777 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.000208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:33525 (local address 127.0.0.1:51778)
0929 03:05:31.001137 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.001149 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.001238 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.001671 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.001675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.001691 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:31.001801 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.001807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.002871 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.002875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.004335 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.004344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.005167 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.005170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.005269 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.005305 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.005429 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.005560 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"mutex_wait_us":186,"thread_start_us":107,"threads_started":1}
I20260929 03:05:31.006630 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.999642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58608 (local address 127.7.209.65:35229)
0929 03:05:31.000694 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.000701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.000724 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:05:31.000798 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.000803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.000965 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.001092 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:31.001852 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.003118 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.004379 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.005230 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.005332 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.005488 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.006314 (+ 826us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.006482 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":938,"thread_start_us":94,"threads_started":1}
I20260929 03:05:31.007037 10253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.999423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:35229 (local address 127.0.0.1:58608)
0929 03:05:31.000152 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.000212 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.000374 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.001248 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.001255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.001287 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:31.001556 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.001564 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.003241 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.003243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.004256 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.004265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.005544 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.005547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.005666 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.005688 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.005763 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.005831 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":163,"spinlock_wait_cycles":10240,"thread_start_us":138,"threads_started":1}
I20260929 03:05:31.007468 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.999540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53666 (local address 127.7.209.66:40031)
0929 03:05:31.000466 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.000480 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.000508 (+ 28us) server_negotiation.cc:407] Connection header received
0929 03:05:31.000596 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.000602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.000865 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.001042 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:31.001824 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.003056 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.005466 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.006323 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.006473 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.006633 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.007179 (+ 546us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.007299 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":115,"threads_started":1}
I20260929 03:05:31.007068 10254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:30.999319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:40031 (local address 127.0.0.1:53666)
0929 03:05:31.000152 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.000220 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.000374 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.001248 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.001255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.001287 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:31.001556 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.001564 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.003190 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.003193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.005338 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.005347 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.006699 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.006702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.006807 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.006828 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.006890 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.006954 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":1031,"spinlock_wait_cycles":9856,"thread_start_us":83,"threads_started":1}
I20260929 03:05:31.009382 10170 tablet_service.cc:1467] Processing CreateTablet for tablet 264014c74a62438595e332fb70428a9c (DEFAULT_TABLE table=test-table [id=1b27da9bf61940eb903695287f966647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:31.010716 10170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 264014c74a62438595e332fb70428a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:31.010252 10030 tablet_service.cc:1467] Processing CreateTablet for tablet 264014c74a62438595e332fb70428a9c (DEFAULT_TABLE table=test-table [id=1b27da9bf61940eb903695287f966647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:31.010250 9890 tablet_service.cc:1467] Processing CreateTablet for tablet 264014c74a62438595e332fb70428a9c (DEFAULT_TABLE table=test-table [id=1b27da9bf61940eb903695287f966647]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:31.011427 10030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 264014c74a62438595e332fb70428a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:31.011427 9890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 264014c74a62438595e332fb70428a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:31.018605 10259 tablet_bootstrap.cc:492] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e: Bootstrap starting.
I20260929 03:05:31.020149 10260 tablet_bootstrap.cc:492] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3: Bootstrap starting.
I20260929 03:05:31.020303 10261 tablet_bootstrap.cc:492] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0: Bootstrap starting.
I20260929 03:05:31.021780 10259 tablet_bootstrap.cc:654] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:31.022516 10260 tablet_bootstrap.cc:654] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:31.022516 10261 tablet_bootstrap.cc:654] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:31.022588 10259 log.cc:824] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:31.023418 10261 log.cc:824] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:31.023418 10260 log.cc:824] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:31.024747 10259 tablet_bootstrap.cc:492] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e: No bootstrap required, opened a new log
I20260929 03:05:31.024940 10259 ts_tablet_manager.cc:1397] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260929 03:05:31.025344 10260 tablet_bootstrap.cc:492] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3: No bootstrap required, opened a new log
I20260929 03:05:31.025542 10260 ts_tablet_manager.cc:1397] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:05:31.025753 10261 tablet_bootstrap.cc:492] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0: No bootstrap required, opened a new log
I20260929 03:05:31.025919 10261 ts_tablet_manager.cc:1397] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 03:05:31.028618 10259 raft_consensus.cc:348] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.028873 10259 raft_consensus.cc:374] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:31.028927 10259 raft_consensus.cc:729] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2daa50738ae7444a982e73f0b39c271e, State: Initialized, Role: FOLLOWER
I20260929 03:05:31.029421 10259 consensus_queue.cc:260] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.030145 10261 raft_consensus.cc:348] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.030328 10236 heartbeater.cc:503] Master 127.7.209.126:39725 was elected leader, sending a full tablet report...
I20260929 03:05:31.030224 10260 raft_consensus.cc:348] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.030431 10260 raft_consensus.cc:374] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:31.030431 10261 raft_consensus.cc:374] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:31.030481 10261 raft_consensus.cc:729] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80e1755b229e41219d14e6b9a14858d0, State: Initialized, Role: FOLLOWER
I20260929 03:05:31.030490 10260 raft_consensus.cc:729] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fedc543ed7248b68b7f4b66b45d55c3, State: Initialized, Role: FOLLOWER
I20260929 03:05:31.030620 10259 ts_tablet_manager.cc:1428] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260929 03:05:31.030966 10261 consensus_queue.cc:260] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.030966 10260 consensus_queue.cc:260] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.031661 9956 heartbeater.cc:503] Master 127.7.209.126:39725 was elected leader, sending a full tablet report...
I20260929 03:05:31.032024 10261 ts_tablet_manager.cc:1428] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 03:05:31.032024 10260 ts_tablet_manager.cc:1428] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 03:05:31.032384 10096 heartbeater.cc:503] Master 127.7.209.126:39725 was elected leader, sending a full tablet report...
I20260929 03:05:31.157491 10266 raft_consensus.cc:482] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:31.157783 10266 raft_consensus.cc:504] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.159196 10266 leader_election.cc:288] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66:40031), 2daa50738ae7444a982e73f0b39c271e (127.7.209.67:33525)
I20260929 03:05:31.163381 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.159645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:46975 (local address 127.7.209.66:40031)
0929 03:05:31.159798 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.159803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.159964 (+ 161us) server_negotiation.cc:407] Connection header received
0929 03:05:31.160144 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.160148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.160213 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.160299 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:31.160597 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.161177 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.162345 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.162818 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.162937 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.162996 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.163168 (+ 172us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.163240 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 03:05:31.163398 10268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.159540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:40031 (local address 127.7.209.65:46975)
0929 03:05:31.159866 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.159881 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.160016 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.160326 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.160329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.160351 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:31.160497 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.160503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.161283 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.161286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.162228 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.162238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.162926 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.162930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.163034 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.163063 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.163143 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.163221 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":141,"threads_started":1}
I20260929 03:05:31.163631 10269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.160026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:33525 (local address 127.7.209.65:50709)
0929 03:05:31.160374 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.160387 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.160484 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.160721 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.160724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.160738 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:31.160864 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.160869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.161594 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.161597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.162408 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.162416 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.163194 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.163198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.163294 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.163318 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.163394 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.163460 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":2,"thread_start_us":138,"threads_started":1}
I20260929 03:05:31.163630 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.160216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:50709 (local address 127.7.209.67:33525)
0929 03:05:31.160359 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.160363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.160460 (+ 97us) server_negotiation.cc:407] Connection header received
0929 03:05:31.160586 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.160589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.160636 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.160703 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:31.160967 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.161491 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.162538 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.163081 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.163203 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.163253 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.163422 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.163502 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 03:05:31.164192 10050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264014c74a62438595e332fb70428a9c" candidate_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80e1755b229e41219d14e6b9a14858d0" is_pre_election: true
I20260929 03:05:31.164290 10190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264014c74a62438595e332fb70428a9c" candidate_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2daa50738ae7444a982e73f0b39c271e" is_pre_election: true
I20260929 03:05:31.164544 10050 raft_consensus.cc:2454] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fedc543ed7248b68b7f4b66b45d55c3 in term 0.
I20260929 03:05:31.164580 10190 raft_consensus.cc:2454] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fedc543ed7248b68b7f4b66b45d55c3 in term 0.
I20260929 03:05:31.165046 9844 leader_election.cc:302] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2daa50738ae7444a982e73f0b39c271e, 3fedc543ed7248b68b7f4b66b45d55c3; no voters:
I20260929 03:05:31.165319 10266 raft_consensus.cc:2781] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:31.165398 10266 raft_consensus.cc:482] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:31.165449 10266 raft_consensus.cc:3037] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:31.166718 10266 raft_consensus.cc:504] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.167042 10266 leader_election.cc:288] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [CANDIDATE]: Term 1 election: Requested vote from peers 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66:40031), 2daa50738ae7444a982e73f0b39c271e (127.7.209.67:33525)
I20260929 03:05:31.167435 10050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264014c74a62438595e332fb70428a9c" candidate_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80e1755b229e41219d14e6b9a14858d0"
I20260929 03:05:31.167483 10190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264014c74a62438595e332fb70428a9c" candidate_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2daa50738ae7444a982e73f0b39c271e"
I20260929 03:05:31.167582 10050 raft_consensus.cc:3037] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:31.167610 10190 raft_consensus.cc:3037] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:31.169103 10050 raft_consensus.cc:2454] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fedc543ed7248b68b7f4b66b45d55c3 in term 1.
I20260929 03:05:31.169338 10190 raft_consensus.cc:2454] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fedc543ed7248b68b7f4b66b45d55c3 in term 1.
I20260929 03:05:31.169430 9844 leader_election.cc:302] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fedc543ed7248b68b7f4b66b45d55c3, 80e1755b229e41219d14e6b9a14858d0; no voters:
I20260929 03:05:31.169593 10266 raft_consensus.cc:2781] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:31.169840 10266 raft_consensus.cc:686] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 LEADER]: Becoming Leader. State: Replica: 3fedc543ed7248b68b7f4b66b45d55c3, State: Running, Role: LEADER
I20260929 03:05:31.170243 10266 consensus_queue.cc:237] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.173974 9753 catalog_manager.cc:5495] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65). New cstate: current_term: 1 leader_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } health_report { overall_health: UNKNOWN } } }
W20260929 03:05:31.190076 10237 tablet.cc:2369] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:05:31.199747 10097 tablet.cc:2369] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:05:31.200990 9957 tablet.cc:2369] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 03:05:31 dist-test-slave-bdps krb5kdc[9708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651130, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:31.216087 10277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.207776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39725 (local address 127.0.0.1:33698)
0929 03:05:31.208083 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.208097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.208248 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.208482 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.208485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.208969 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:31.209091 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.209098 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.209925 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.209928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.210506 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.210513 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.210923 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.210926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.211017 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.211616 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:31.211628 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:31.213185 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:31.214666 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:31.214675 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:31.214681 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:31.214910 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:31.215118 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:31.215120 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:31.215121 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:31.215200 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:31.215497 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:31.215504 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:31.215653 (+ 149us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.215767 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.215916 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":110,"threads_started":1}
I20260929 03:05:31.216099 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.207807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.7.209.126:39725)
0929 03:05:31.207927 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.207932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.208195 (+ 263us) server_negotiation.cc:407] Connection header received
0929 03:05:31.208334 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.208336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.208378 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.208447 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:31.209209 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.209817 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.210605 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.210825 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.210913 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.213310 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:31.213313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:31.213316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:31.213338 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:31.214570 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:31.214984 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:31.214987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:31.214988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:31.215053 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:31.215262 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:31.215264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:31.215265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:31.215426 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:31.215495 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.215812 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.215981 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":36}
I20260929 03:05:31.220186 9753 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:33698:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 03:05:31.220988 9753 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:31.225708 10030 tablet_service.cc:1467] Processing CreateTablet for tablet 3c58fda31eea4777b022e6c4e6c19357 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f9973e23bd341cca61c615a6af45875]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 03:05:31.225723 9890 tablet_service.cc:1467] Processing CreateTablet for tablet 3c58fda31eea4777b022e6c4e6c19357 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f9973e23bd341cca61c615a6af45875]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 03:05:31.225798 10170 tablet_service.cc:1467] Processing CreateTablet for tablet 3c58fda31eea4777b022e6c4e6c19357 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f9973e23bd341cca61c615a6af45875]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 03:05:31.226564 10030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c58fda31eea4777b022e6c4e6c19357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:31.226565 9890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c58fda31eea4777b022e6c4e6c19357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:31.226567 10170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c58fda31eea4777b022e6c4e6c19357. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:31.229651 10261 tablet_bootstrap.cc:492] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0: Bootstrap starting.
I20260929 03:05:31.229722 10259 tablet_bootstrap.cc:492] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: Bootstrap starting.
I20260929 03:05:31.229722 10260 tablet_bootstrap.cc:492] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3: Bootstrap starting.
I20260929 03:05:31.230671 10261 tablet_bootstrap.cc:654] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:31.230814 10259 tablet_bootstrap.cc:654] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:31.231238 10260 tablet_bootstrap.cc:654] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:31.232036 10261 tablet_bootstrap.cc:492] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0: No bootstrap required, opened a new log
I20260929 03:05:31.232136 10261 ts_tablet_manager.cc:1397] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 03:05:31.232630 10260 tablet_bootstrap.cc:492] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3: No bootstrap required, opened a new log
I20260929 03:05:31.232648 10259 tablet_bootstrap.cc:492] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: No bootstrap required, opened a new log
I20260929 03:05:31.232709 10259 ts_tablet_manager.cc:1397] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 03:05:31.232704 10260 ts_tablet_manager.cc:1397] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 03:05:31.232962 10261 raft_consensus.cc:348] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.233078 10261 raft_consensus.cc:374] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:31.233110 10261 raft_consensus.cc:729] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80e1755b229e41219d14e6b9a14858d0, State: Initialized, Role: FOLLOWER
I20260929 03:05:31.233071 10259 raft_consensus.cc:348] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.233158 10259 raft_consensus.cc:374] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:31.233179 10259 raft_consensus.cc:729] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2daa50738ae7444a982e73f0b39c271e, State: Initialized, Role: FOLLOWER
I20260929 03:05:31.233143 10260 raft_consensus.cc:348] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.233245 10260 raft_consensus.cc:374] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:31.233280 10260 raft_consensus.cc:729] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fedc543ed7248b68b7f4b66b45d55c3, State: Initialized, Role: FOLLOWER
I20260929 03:05:31.233251 10261 consensus_queue.cc:260] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.233291 10259 consensus_queue.cc:260] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.233403 10260 consensus_queue.cc:260] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.233548 10259 ts_tablet_manager.cc:1428] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 03:05:31.233695 10260 ts_tablet_manager.cc:1428] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260929 03:05:31.233800 10266 tablet_replica.cc:439] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } } }
I20260929 03:05:31.233853 10265 tablet_replica.cc:439] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } } }
I20260929 03:05:31.233953 10266 tablet_replica.cc:442] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:31.233991 10265 tablet_replica.cc:442] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:31.234499 10267 tablet_replica.cc:439] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } } }
I20260929 03:05:31.234658 10267 tablet_replica.cc:442] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:31.234514 10261 ts_tablet_manager.cc:1428] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 03:05:31.347410 10265 raft_consensus.cc:482] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:31.347649 10265 raft_consensus.cc:504] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.349171 10265 leader_election.cc:288] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229), 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66:40031)
I20260929 03:05:31.354694 10255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.349582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:38563 (local address 127.7.209.66:40031)
0929 03:05:31.350027 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.350031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.350044 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:05:31.350102 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.350105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.350159 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.350251 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:31.350535 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.351395 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.353232 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.353934 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.354059 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.354121 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.354435 (+ 314us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.354523 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342}
I20260929 03:05:31.354686 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.349589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:51589 (local address 127.7.209.65:35229)
0929 03:05:31.349776 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.349780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.350104 (+ 324us) server_negotiation.cc:407] Connection header received
0929 03:05:31.350226 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.350229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.350284 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.350358 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:31.350644 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.351416 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.353232 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.353930 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.354059 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.354121 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.354438 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.354527 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260929 03:05:31.354696 10129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.349445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:40031 (local address 127.7.209.67:38563)
0929 03:05:31.349573 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.349591 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.349876 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.350255 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.350257 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.350268 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:31.350419 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.350425 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.351558 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.351563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.353081 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.353096 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.354055 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.354060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.354279 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.354334 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.354416 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.354531 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":127}
I20260929 03:05:31.354708 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.349445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:35229 (local address 127.7.209.67:51589)
0929 03:05:31.350000 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.350013 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.350120 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.350389 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.350392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.350412 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:31.350543 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.350549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.351579 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.351582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.353081 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.353096 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.354067 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.354070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.354261 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.354340 (+ 79us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.354421 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.354524 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":282,"thread_start_us":149,"threads_started":1}
I20260929 03:05:31.355295 10050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c58fda31eea4777b022e6c4e6c19357" candidate_uuid: "2daa50738ae7444a982e73f0b39c271e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80e1755b229e41219d14e6b9a14858d0" is_pre_election: true
I20260929 03:05:31.355501 10050 raft_consensus.cc:2454] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2daa50738ae7444a982e73f0b39c271e in term 0.
I20260929 03:05:31.355656 9910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c58fda31eea4777b022e6c4e6c19357" candidate_uuid: "2daa50738ae7444a982e73f0b39c271e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" is_pre_election: true
I20260929 03:05:31.355913 9910 raft_consensus.cc:2454] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2daa50738ae7444a982e73f0b39c271e in term 0.
I20260929 03:05:31.355942 10124 leader_election.cc:302] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2daa50738ae7444a982e73f0b39c271e, 80e1755b229e41219d14e6b9a14858d0; no voters:
I20260929 03:05:31.356317 10265 raft_consensus.cc:2781] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:31.356389 10265 raft_consensus.cc:482] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:31.356432 10265 raft_consensus.cc:3037] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:31.357741 10265 raft_consensus.cc:504] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.358114 10265 leader_election.cc:288] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [CANDIDATE]: Term 1 election: Requested vote from peers 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229), 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66:40031)
I20260929 03:05:31.358438 9910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c58fda31eea4777b022e6c4e6c19357" candidate_uuid: "2daa50738ae7444a982e73f0b39c271e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fedc543ed7248b68b7f4b66b45d55c3"
I20260929 03:05:31.358482 10050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c58fda31eea4777b022e6c4e6c19357" candidate_uuid: "2daa50738ae7444a982e73f0b39c271e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80e1755b229e41219d14e6b9a14858d0"
I20260929 03:05:31.358562 9910 raft_consensus.cc:3037] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:31.358618 10050 raft_consensus.cc:3037] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:31.359658 9910 raft_consensus.cc:2454] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2daa50738ae7444a982e73f0b39c271e in term 1.
I20260929 03:05:31.359767 10050 raft_consensus.cc:2454] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2daa50738ae7444a982e73f0b39c271e in term 1.
I20260929 03:05:31.359964 10121 leader_election.cc:302] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2daa50738ae7444a982e73f0b39c271e, 3fedc543ed7248b68b7f4b66b45d55c3; no voters:
I20260929 03:05:31.360126 10265 raft_consensus.cc:2781] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:31.360392 10265 raft_consensus.cc:686] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [term 1 LEADER]: Becoming Leader. State: Replica: 2daa50738ae7444a982e73f0b39c271e, State: Running, Role: LEADER
I20260929 03:05:31.360785 10265 consensus_queue.cc:237] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.363265 10282 tablet_replica.cc:439] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: TxnStatusTablet state changed. Reason: New leader 2daa50738ae7444a982e73f0b39c271e. Latest consensus state: current_term: 1 leader_uuid: "2daa50738ae7444a982e73f0b39c271e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } } }
I20260929 03:05:31.363389 10282 tablet_replica.cc:442] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: This TxnStatusTablet replica's current role is: LEADER
I20260929 03:05:31.363684 10284 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 03:05:31.363906 10284 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 03:05:31.364830 9750 catalog_manager.cc:5495] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2daa50738ae7444a982e73f0b39c271e (127.7.209.67). New cstate: current_term: 1 leader_uuid: "2daa50738ae7444a982e73f0b39c271e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } health_report { overall_health: HEALTHY } } }
I20260929 03:05:31.387501 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.383756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51794 (local address 127.7.209.67:33525)
0929 03:05:31.383873 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.383877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.383893 (+ 16us) server_negotiation.cc:407] Connection header received
0929 03:05:31.384110 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.384114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.384186 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.384275 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:31.384529 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.385056 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.385976 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.386215 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.386303 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.386584 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:31.387066 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:31.387179 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.387260 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.387318 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 03:05:31.387477 10277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.383653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:33525 (local address 127.0.0.1:51794)
0929 03:05:31.383833 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.383850 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.383964 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.384305 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.384308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.384317 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 03:05:31.384432 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.384438 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.385165 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.385168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.385865 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.385875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.386312 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.386316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.386400 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.386449 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 03:05:31.387154 (+ 705us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 03:05:31.387171 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.387239 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.387344 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 03:05:31.393976 10050 raft_consensus.cc:1261] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Refusing update from remote peer 2daa50738ae7444a982e73f0b39c271e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:31.393975 9910 raft_consensus.cc:1261] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Refusing update from remote peer 2daa50738ae7444a982e73f0b39c271e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:31.394719 10282 consensus_queue.cc:1035] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [LEADER]: Connected to new peer: Peer: permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:31.394930 10265 consensus_queue.cc:1035] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:31.398825 10266 tablet_replica.cc:439] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3: TxnStatusTablet state changed. Reason: New leader 2daa50738ae7444a982e73f0b39c271e. Latest consensus state: current_term: 1 leader_uuid: "2daa50738ae7444a982e73f0b39c271e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } } }
I20260929 03:05:31.398965 10266 tablet_replica.cc:442] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:31.399518 10267 tablet_replica.cc:439] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0: TxnStatusTablet state changed. Reason: New leader 2daa50738ae7444a982e73f0b39c271e. Latest consensus state: current_term: 1 leader_uuid: "2daa50738ae7444a982e73f0b39c271e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } } }
I20260929 03:05:31.399655 10267 tablet_replica.cc:442] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:31.402952 10265 tablet_replica.cc:439] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2daa50738ae7444a982e73f0b39c271e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } } }
I20260929 03:05:31.403072 10265 tablet_replica.cc:442] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: This TxnStatusTablet replica's current role is: LEADER
I20260929 03:05:31.404330 10289 mvcc.cc:205] Tried to move back new op lower bound from 7334507034180845568 to 7334507034056785920. Current Snapshot: MvccSnapshot[applied={T|T < 7334507034180845568}]
I20260929 03:05:31.404500 10266 tablet_replica.cc:439] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2daa50738ae7444a982e73f0b39c271e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } } }
I20260929 03:05:31.404595 10266 tablet_replica.cc:442] T 3c58fda31eea4777b022e6c4e6c19357 P 3fedc543ed7248b68b7f4b66b45d55c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:31.405865 10282 tablet_replica.cc:439] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2daa50738ae7444a982e73f0b39c271e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } } }
I20260929 03:05:31.405978 10282 tablet_replica.cc:442] T 3c58fda31eea4777b022e6c4e6c19357 P 2daa50738ae7444a982e73f0b39c271e: This TxnStatusTablet replica's current role is: LEADER
I20260929 03:05:31.406955 10288 mvcc.cc:205] Tried to move back new op lower bound from 7334507034180845568 to 7334507034056785920. Current Snapshot: MvccSnapshot[applied={T|T < 7334507034180845568}]
I20260929 03:05:31.408021 10267 tablet_replica.cc:439] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2daa50738ae7444a982e73f0b39c271e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } } }
I20260929 03:05:31.408141 10267 tablet_replica.cc:442] T 3c58fda31eea4777b022e6c4e6c19357 P 80e1755b229e41219d14e6b9a14858d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:31.414746 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.411180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58618 (local address 127.7.209.65:35229)
0929 03:05:31.411310 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.411315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.411327 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:05:31.411398 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.411401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.411453 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.411540 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:31.411808 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.412361 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.413250 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.413489 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.413578 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.413892 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:31.414361 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:31.414466 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.414536 (+ 70us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.414596 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 03:05:31.426844 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.423308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:60487 (local address 127.7.209.67:33525)
0929 03:05:31.423446 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.423451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.423569 (+ 118us) server_negotiation.cc:407] Connection header received
0929 03:05:31.423721 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.423725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.423781 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.423843 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:31.424103 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.424647 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.425705 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.425951 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.426043 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.426366 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:31.426447 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:31.426504 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.426604 (+ 100us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.426683 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:05:31.426854 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.423173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:33525 (local address 127.7.209.65:60487)
0929 03:05:31.423483 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.423497 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.423605 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.423858 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.423861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.423870 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 03:05:31.423997 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.424002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.424843 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.424847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.425567 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.425577 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.426052 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.426055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.426161 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.426208 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 03:05:31.426518 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 03:05:31.426525 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.426583 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.426675 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":113,"threads_started":1}
I20260929 03:05:31.435789 10050 raft_consensus.cc:1261] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Refusing update from remote peer 3fedc543ed7248b68b7f4b66b45d55c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:31.436002 10190 raft_consensus.cc:1261] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Refusing update from remote peer 3fedc543ed7248b68b7f4b66b45d55c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:31.436352 10266 consensus_queue.cc:1035] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:31.436570 10270 consensus_queue.cc:1035] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:31.445895 10287 mvcc.cc:205] Tried to move back new op lower bound from 7334507034355822592 to 7334507033276424192. Current Snapshot: MvccSnapshot[applied={T|T < 7334507034355822592}]
I20260929 03:05:31.469591 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.461691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:35229 (local address 127.7.209.67:55779)
0929 03:05:31.465825 (+ 4134us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.465839 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.465949 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.466175 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.466178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.466187 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 03:05:31.466299 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.466306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.467179 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.467182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.468386 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.468396 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.468842 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.468845 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.468934 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.468987 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 03:05:31.469271 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 03:05:31.469279 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.469341 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.469429 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4014,"thread_start_us":68,"threads_started":1}
I20260929 03:05:31.469573 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.461756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:55779 (local address 127.7.209.65:35229)
0929 03:05:31.461987 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.461991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.465914 (+ 3923us) server_negotiation.cc:407] Connection header received
0929 03:05:31.466036 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.466040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.466090 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.466160 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:31.466393 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.467038 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.468503 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.468747 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.468836 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.469113 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:31.469200 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:31.469259 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.469350 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.469411 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260929 03:05:31.499485 9751 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:31.499751 9751 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:31.502900 9751 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea: Sending DeleteTablet for 3 replicas of tablet 264014c74a62438595e332fb70428a9c
I20260929 03:05:31.503671 10030 tablet_service.cc:1514] Processing DeleteTablet for tablet 264014c74a62438595e332fb70428a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:31 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:53666
I20260929 03:05:31.503814 9890 tablet_service.cc:1514] Processing DeleteTablet for tablet 264014c74a62438595e332fb70428a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:31 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:58608
I20260929 03:05:31.504472 10308 tablet_replica.cc:330] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0: stopping tablet replica
I20260929 03:05:31.504669 10170 tablet_service.cc:1514] Processing DeleteTablet for tablet 264014c74a62438595e332fb70428a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:31 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:51778
I20260929 03:05:31.504988 10308 raft_consensus.cc:2229] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:31.505290 10308 raft_consensus.cc:2258] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:31.506124 9750 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:05:31.506601 9750 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:31.507484 10308 ts_tablet_manager.cc:1905] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:31.511708 10308 ts_tablet_manager.cc:1918] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:05:31.511963 10308 log.cc:1196] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/264014c74a62438595e332fb70428a9c
I20260929 03:05:31.512508 10030 tablet_service.cc:1467] Processing CreateTablet for tablet 26fd7e3f0a334301a9d2b06470b26fda (DEFAULT_TABLE table=test-table [id=d2b63342ba89409d8844c8eb7a667d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:31.512710 9890 tablet_service.cc:1467] Processing CreateTablet for tablet 26fd7e3f0a334301a9d2b06470b26fda (DEFAULT_TABLE table=test-table [id=d2b63342ba89409d8844c8eb7a667d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:31.512840 10030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26fd7e3f0a334301a9d2b06470b26fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:31.513007 9890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26fd7e3f0a334301a9d2b06470b26fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:31.515703 10261 tablet_bootstrap.cc:492] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0: Bootstrap starting.
I20260929 03:05:31.515790 10170 tablet_service.cc:1467] Processing CreateTablet for tablet 26fd7e3f0a334301a9d2b06470b26fda (DEFAULT_TABLE table=test-table [id=d2b63342ba89409d8844c8eb7a667d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:31.516105 10170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26fd7e3f0a334301a9d2b06470b26fda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:31.516851 10261 tablet_bootstrap.cc:654] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:31.518110 10308 ts_tablet_manager.cc:1939] T 264014c74a62438595e332fb70428a9c P 80e1755b229e41219d14e6b9a14858d0: Deleting consensus metadata
I20260929 03:05:31.518981 10259 tablet_bootstrap.cc:492] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e: Bootstrap starting.
I20260929 03:05:31.519914 9740 catalog_manager.cc:4841] TS 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66:40031): tablet 264014c74a62438595e332fb70428a9c (table test-table [id=1b27da9bf61940eb903695287f966647]) successfully deleted
I20260929 03:05:31.519986 10259 tablet_bootstrap.cc:654] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:31.521039 10261 tablet_bootstrap.cc:492] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0: No bootstrap required, opened a new log
I20260929 03:05:31.521132 10261 ts_tablet_manager.cc:1397] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260929 03:05:31.521580 10261 raft_consensus.cc:348] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.521733 10261 raft_consensus.cc:374] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:31.521765 10261 raft_consensus.cc:729] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80e1755b229e41219d14e6b9a14858d0, State: Initialized, Role: FOLLOWER
I20260929 03:05:31.521893 10261 consensus_queue.cc:260] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.522156 10261 ts_tablet_manager.cc:1428] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:31.522254 10260 tablet_bootstrap.cc:492] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3: Bootstrap starting.
I20260929 03:05:31.523223 10260 tablet_bootstrap.cc:654] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:31.523355 10310 tablet_replica.cc:330] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e: stopping tablet replica
I20260929 03:05:31.523772 10310 raft_consensus.cc:2229] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:31.523983 10310 raft_consensus.cc:2258] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:31.525445 10310 ts_tablet_manager.cc:1905] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:31.525734 10309 tablet_replica.cc:330] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3: stopping tablet replica
I20260929 03:05:31.526094 10309 raft_consensus.cc:2229] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:05:31.525713 10259 tablet_bootstrap.cc:492] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e: No bootstrap required, opened a new log
I20260929 03:05:31.526257 10259 ts_tablet_manager.cc:1397] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260929 03:05:31.526683 10259 raft_consensus.cc:348] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.526793 10259 raft_consensus.cc:374] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:31.526819 10259 raft_consensus.cc:729] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2daa50738ae7444a982e73f0b39c271e, State: Initialized, Role: FOLLOWER
I20260929 03:05:31.526929 10259 consensus_queue.cc:260] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.527164 10259 ts_tablet_manager.cc:1428] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:31.530534 10310 ts_tablet_manager.cc:1918] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:05:31.530632 10310 log.cc:1196] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/264014c74a62438595e332fb70428a9c
I20260929 03:05:31.531390 10310 ts_tablet_manager.cc:1939] T 264014c74a62438595e332fb70428a9c P 2daa50738ae7444a982e73f0b39c271e: Deleting consensus metadata
I20260929 03:05:31.527383 10260 tablet_bootstrap.cc:492] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3: No bootstrap required, opened a new log
I20260929 03:05:31.532554 10260 ts_tablet_manager.cc:1397] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260929 03:05:31.532584 9740 catalog_manager.cc:4841] TS 2daa50738ae7444a982e73f0b39c271e (127.7.209.67:33525): tablet 264014c74a62438595e332fb70428a9c (table test-table [id=1b27da9bf61940eb903695287f966647]) successfully deleted
I20260929 03:05:31.533034 10260 raft_consensus.cc:348] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.533152 10260 raft_consensus.cc:374] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:31.533180 10260 raft_consensus.cc:729] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fedc543ed7248b68b7f4b66b45d55c3, State: Initialized, Role: FOLLOWER
I20260929 03:05:31.533293 10260 consensus_queue.cc:260] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.533536 10260 ts_tablet_manager.cc:1428] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:31.534442 10309 raft_consensus.cc:2258] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:31.535879 10309 ts_tablet_manager.cc:1905] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:31.539109 10309 ts_tablet_manager.cc:1918] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:05:31.539209 10309 log.cc:1196] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/wal/wals/264014c74a62438595e332fb70428a9c
I20260929 03:05:31.539530 10309 ts_tablet_manager.cc:1939] T 264014c74a62438595e332fb70428a9c P 3fedc543ed7248b68b7f4b66b45d55c3: Deleting consensus metadata
I20260929 03:05:31.540738 9737 catalog_manager.cc:4841] TS 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229): tablet 264014c74a62438595e332fb70428a9c (table test-table [id=1b27da9bf61940eb903695287f966647]) successfully deleted
I20260929 03:05:31.838411 10267 raft_consensus.cc:482] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:31.838600 10267 raft_consensus.cc:504] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.840052 10267 leader_election.cc:288] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229), 2daa50738ae7444a982e73f0b39c271e (127.7.209.67:33525)
I20260929 03:05:31.845763 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.840542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:50843 (local address 127.7.209.67:33525)
0929 03:05:31.840787 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.840791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.840925 (+ 134us) server_negotiation.cc:407] Connection header received
0929 03:05:31.841133 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.841136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.841196 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.841265 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:31.841679 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.842520 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.844365 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.845023 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.845159 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.845224 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.845498 (+ 274us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.845587 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260929 03:05:31.845834 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.840497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:43151 (local address 127.7.209.65:35229)
0929 03:05:31.840704 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.840709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.840930 (+ 221us) server_negotiation.cc:407] Connection header received
0929 03:05:31.841097 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.841101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.841171 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.841265 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:31.841685 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.842518 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.844365 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.845055 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.845159 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.845224 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.845594 (+ 370us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.845721 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 03:05:31.845968 10315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.840390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:33525 (local address 127.7.209.66:50843)
0929 03:05:31.840778 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.840794 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.840966 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.841288 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.841292 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.841321 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:31.841522 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.841527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.842648 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.842652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.844246 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.844258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.845153 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.845158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.845356 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.845391 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.845480 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.845574 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":164,"spinlock_wait_cycles":39424,"thread_start_us":98,"threads_started":1}
I20260929 03:05:31.846211 10314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.840390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:35229 (local address 127.7.209.66:43151)
0929 03:05:31.840778 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.840794 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.840966 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.841288 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.841291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.841319 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:31.841519 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.841527 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.842623 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.842628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.844251 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.844258 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.845154 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.845158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.845480 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.845500 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.845566 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.845648 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":111,"spinlock_wait_cycles":15104,"thread_start_us":102,"threads_started":1}
I20260929 03:05:31.846673 9910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fd7e3f0a334301a9d2b06470b26fda" candidate_uuid: "80e1755b229e41219d14e6b9a14858d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" is_pre_election: true
I20260929 03:05:31.846673 10190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fd7e3f0a334301a9d2b06470b26fda" candidate_uuid: "80e1755b229e41219d14e6b9a14858d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2daa50738ae7444a982e73f0b39c271e" is_pre_election: true
I20260929 03:05:31.846854 9910 raft_consensus.cc:2454] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80e1755b229e41219d14e6b9a14858d0 in term 0.
I20260929 03:05:31.846854 10190 raft_consensus.cc:2454] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80e1755b229e41219d14e6b9a14858d0 in term 0.
I20260929 03:05:31.847216 9984 leader_election.cc:302] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2daa50738ae7444a982e73f0b39c271e, 80e1755b229e41219d14e6b9a14858d0; no voters:
I20260929 03:05:31.847483 10267 raft_consensus.cc:2781] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:31.847555 10267 raft_consensus.cc:482] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:31.847602 10267 raft_consensus.cc:3037] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:31.848817 10267 raft_consensus.cc:504] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.849184 10267 leader_election.cc:288] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [CANDIDATE]: Term 1 election: Requested vote from peers 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229), 2daa50738ae7444a982e73f0b39c271e (127.7.209.67:33525)
I20260929 03:05:31.849527 10190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fd7e3f0a334301a9d2b06470b26fda" candidate_uuid: "80e1755b229e41219d14e6b9a14858d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2daa50738ae7444a982e73f0b39c271e"
I20260929 03:05:31.849527 9910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26fd7e3f0a334301a9d2b06470b26fda" candidate_uuid: "80e1755b229e41219d14e6b9a14858d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fedc543ed7248b68b7f4b66b45d55c3"
I20260929 03:05:31.849685 10190 raft_consensus.cc:3037] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:31.849709 9910 raft_consensus.cc:3037] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:31.851076 10190 raft_consensus.cc:2454] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80e1755b229e41219d14e6b9a14858d0 in term 1.
I20260929 03:05:31.851076 9910 raft_consensus.cc:2454] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80e1755b229e41219d14e6b9a14858d0 in term 1.
I20260929 03:05:31.851382 9984 leader_election.cc:302] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2daa50738ae7444a982e73f0b39c271e, 80e1755b229e41219d14e6b9a14858d0; no voters:
I20260929 03:05:31.851548 10267 raft_consensus.cc:2781] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:31.851806 10267 raft_consensus.cc:686] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 1 LEADER]: Becoming Leader. State: Replica: 80e1755b229e41219d14e6b9a14858d0, State: Running, Role: LEADER
I20260929 03:05:31.852114 10267 consensus_queue.cc:237] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:31.854959 9751 catalog_manager.cc:5495] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66). New cstate: current_term: 1 leader_uuid: "80e1755b229e41219d14e6b9a14858d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } health_report { overall_health: UNKNOWN } } }
I20260929 03:05:31.904888 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.901059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53676 (local address 127.7.209.66:40031)
0929 03:05:31.901334 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.901337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.901349 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:05:31.901427 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.901429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.901486 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.901595 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:31.901843 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.902468 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.903388 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.903623 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.903706 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.903901 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:31.904507 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:31.904601 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.904644 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.904694 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":116,"threads_started":1}
I20260929 03:05:31.916775 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.913056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:57119 (local address 127.7.209.67:33525)
0929 03:05:31.913220 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.913225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.913375 (+ 150us) server_negotiation.cc:407] Connection header received
0929 03:05:31.913540 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.913542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.913596 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.913694 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:31.913975 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.914537 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.915613 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.915853 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.915944 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.916289 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:31.916373 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:31.916429 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.916541 (+ 112us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.916600 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 03:05:31.916821 10321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.912895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:33525 (local address 127.7.209.66:57119)
0929 03:05:31.913284 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.913300 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.913429 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.913717 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.913721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.913729 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 03:05:31.913868 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.913874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.914654 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.914658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.915488 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.915498 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.915958 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.915961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.916096 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.916150 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 03:05:31.916449 (+ 299us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 03:05:31.916456 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.916519 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.916606 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":155,"threads_started":1}
I20260929 03:05:31.923858 9910 raft_consensus.cc:1261] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Refusing update from remote peer 80e1755b229e41219d14e6b9a14858d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:31.923866 10190 raft_consensus.cc:1261] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Refusing update from remote peer 80e1755b229e41219d14e6b9a14858d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:31.924389 10267 consensus_queue.cc:1035] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:31.924595 10316 consensus_queue.cc:1035] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:31.948818 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.943682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:40031 (local address 127.7.209.67:43853)
0929 03:05:31.943876 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:31.943889 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:31.943956 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:31.945916 (+ 1960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:31.945919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:31.945926 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 03:05:31.946033 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.946039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.946995 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.946998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.947663 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:31.947673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.948221 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.948223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:31.948289 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.948325 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 03:05:31.948578 (+ 253us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 03:05:31.948582 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 03:05:31.948619 (+ 37us) client_negotiation.cc:240] Negotiation successful
0929 03:05:31.948714 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260929 03:05:31.948833 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:31.943747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:43853 (local address 127.7.209.66:40031)
0929 03:05:31.945727 (+ 1980us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:31.945730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:31.945741 (+ 11us) server_negotiation.cc:407] Connection header received
0929 03:05:31.945777 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:31.945780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:31.945819 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:31.945888 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:31.946138 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.946905 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.947770 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:31.948135 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:31.948231 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:31.948423 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:31.948512 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:31.948568 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:31.948640 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 03:05:31.948698 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1905}
W20260929 03:05:31.949586 10288 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 03:05:33.507849 9753 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:33.508050 9753 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:33.510748 9753 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea: Sending DeleteTablet for 3 replicas of tablet 26fd7e3f0a334301a9d2b06470b26fda
I20260929 03:05:33.511397 10030 tablet_service.cc:1514] Processing DeleteTablet for tablet 26fd7e3f0a334301a9d2b06470b26fda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:33 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:53666
I20260929 03:05:33.511435 9890 tablet_service.cc:1514] Processing DeleteTablet for tablet 26fd7e3f0a334301a9d2b06470b26fda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:33 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:58608
I20260929 03:05:33.511791 10170 tablet_service.cc:1514] Processing DeleteTablet for tablet 26fd7e3f0a334301a9d2b06470b26fda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:33 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:51778
I20260929 03:05:33.511842 10348 tablet_replica.cc:330] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3: stopping tablet replica
I20260929 03:05:33.511992 10348 raft_consensus.cc:2229] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:33.512141 10348 raft_consensus.cc:2258] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:33.512300 10349 tablet_replica.cc:330] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e: stopping tablet replica
I20260929 03:05:33.512512 10349 raft_consensus.cc:2229] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:33.512692 10349 raft_consensus.cc:2258] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:33.513043 10348 ts_tablet_manager.cc:1905] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:33.513427 10349 ts_tablet_manager.cc:1905] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:33.513819 10347 tablet_replica.cc:330] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0: stopping tablet replica
I20260929 03:05:33.514014 10347 raft_consensus.cc:2229] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:05:33.514432 10347 raft_consensus.cc:2258] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:33.515093 10347 ts_tablet_manager.cc:1905] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:33.516685 10348 ts_tablet_manager.cc:1918] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 03:05:33.516781 10349 ts_tablet_manager.cc:1918] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 03:05:33.516824 10348 log.cc:1196] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/wal/wals/26fd7e3f0a334301a9d2b06470b26fda
I20260929 03:05:33.516976 10349 log.cc:1196] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/26fd7e3f0a334301a9d2b06470b26fda
I20260929 03:05:33.517102 10348 ts_tablet_manager.cc:1939] T 26fd7e3f0a334301a9d2b06470b26fda P 3fedc543ed7248b68b7f4b66b45d55c3: Deleting consensus metadata
I20260929 03:05:33.517392 10349 ts_tablet_manager.cc:1939] T 26fd7e3f0a334301a9d2b06470b26fda P 2daa50738ae7444a982e73f0b39c271e: Deleting consensus metadata
I20260929 03:05:33.517983 9737 catalog_manager.cc:4841] TS 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229): tablet 26fd7e3f0a334301a9d2b06470b26fda (table test-table [id=d2b63342ba89409d8844c8eb7a667d34]) successfully deleted
I20260929 03:05:33.518062 10347 ts_tablet_manager.cc:1918] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 03:05:33.518131 10347 log.cc:1196] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/26fd7e3f0a334301a9d2b06470b26fda
I20260929 03:05:33.518198 9740 catalog_manager.cc:4841] TS 2daa50738ae7444a982e73f0b39c271e (127.7.209.67:33525): tablet 26fd7e3f0a334301a9d2b06470b26fda (table test-table [id=d2b63342ba89409d8844c8eb7a667d34]) successfully deleted
I20260929 03:05:33.518407 10347 ts_tablet_manager.cc:1939] T 26fd7e3f0a334301a9d2b06470b26fda P 80e1755b229e41219d14e6b9a14858d0: Deleting consensus metadata
I20260929 03:05:33.519080 9740 catalog_manager.cc:4841] TS 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66:40031): tablet 26fd7e3f0a334301a9d2b06470b26fda (table test-table [id=d2b63342ba89409d8844c8eb7a667d34]) successfully deleted
Sep 29 03:05:33 dist-test-slave-bdps krb5kdc[9708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651133, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 03:05:33 dist-test-slave-bdps krb5kdc[9708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651133, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:33.546099 10357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:33.534050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33706 (local address 127.7.209.126:39725)
0929 03:05:33.534355 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:33.534362 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:33.534377 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:05:33.534476 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:33.534478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:33.534527 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:33.534611 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:33.535358 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:33.535960 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:33.536751 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:33.536986 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:33.537056 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:33.542978 (+ 5922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:33.542981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:33.542984 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:33.543010 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:33.544533 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:33.544963 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:33.544966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:33.544968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:33.545052 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:33.545255 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:33.545257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:33.545258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:33.545449 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:33.545551 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:33.545779 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 03:05:33.545894 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":112,"threads_started":1}
I20260929 03:05:33.548834 9753 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:05:33.549342 9753 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:33.554790 10030 tablet_service.cc:1467] Processing CreateTablet for tablet 652c2017ff3f4758928c7d6e68290770 (DEFAULT_TABLE table=test-table [id=8fd0cdde30b54026b7f59a72eb100735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:33.554829 9890 tablet_service.cc:1467] Processing CreateTablet for tablet 652c2017ff3f4758928c7d6e68290770 (DEFAULT_TABLE table=test-table [id=8fd0cdde30b54026b7f59a72eb100735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:33.554862 10170 tablet_service.cc:1467] Processing CreateTablet for tablet 652c2017ff3f4758928c7d6e68290770 (DEFAULT_TABLE table=test-table [id=8fd0cdde30b54026b7f59a72eb100735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:33.555244 9890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 652c2017ff3f4758928c7d6e68290770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:33.555244 10170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 652c2017ff3f4758928c7d6e68290770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:33.555249 10030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 652c2017ff3f4758928c7d6e68290770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:33.558059 10358 tablet_bootstrap.cc:492] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0: Bootstrap starting.
I20260929 03:05:33.558588 10360 tablet_bootstrap.cc:492] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3: Bootstrap starting.
I20260929 03:05:33.559023 10359 tablet_bootstrap.cc:492] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e: Bootstrap starting.
I20260929 03:05:33.559083 10358 tablet_bootstrap.cc:654] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:33.559544 10360 tablet_bootstrap.cc:654] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:33.560220 10358 tablet_bootstrap.cc:492] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0: No bootstrap required, opened a new log
I20260929 03:05:33.560240 10359 tablet_bootstrap.cc:654] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:33.560322 10358 ts_tablet_manager.cc:1397] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 03:05:33.560822 10360 tablet_bootstrap.cc:492] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3: No bootstrap required, opened a new log
I20260929 03:05:33.560778 10358 raft_consensus.cc:348] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.560958 10358 raft_consensus.cc:374] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:33.561015 10358 raft_consensus.cc:729] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80e1755b229e41219d14e6b9a14858d0, State: Initialized, Role: FOLLOWER
I20260929 03:05:33.561060 10360 ts_tablet_manager.cc:1397] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 03:05:33.561209 10358 consensus_queue.cc:260] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.561519 10358 ts_tablet_manager.cc:1428] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:33.561501 10360 raft_consensus.cc:348] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.561650 10360 raft_consensus.cc:374] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:33.561687 10360 raft_consensus.cc:729] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fedc543ed7248b68b7f4b66b45d55c3, State: Initialized, Role: FOLLOWER
I20260929 03:05:33.561862 10360 consensus_queue.cc:260] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.562306 10360 ts_tablet_manager.cc:1428] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 03:05:33.563225 10359 tablet_bootstrap.cc:492] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e: No bootstrap required, opened a new log
I20260929 03:05:33.563316 10359 ts_tablet_manager.cc:1397] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260929 03:05:33.563752 10359 raft_consensus.cc:348] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.563871 10359 raft_consensus.cc:374] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:33.563925 10359 raft_consensus.cc:729] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2daa50738ae7444a982e73f0b39c271e, State: Initialized, Role: FOLLOWER
I20260929 03:05:33.564070 10359 consensus_queue.cc:260] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.564603 10339 raft_consensus.cc:482] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:33.564702 10339 raft_consensus.cc:504] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.564826 10359 ts_tablet_manager.cc:1428] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:33.565097 10339 leader_election.cc:288] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229), 2daa50738ae7444a982e73f0b39c271e (127.7.209.67:33525)
I20260929 03:05:33.565475 10190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "652c2017ff3f4758928c7d6e68290770" candidate_uuid: "80e1755b229e41219d14e6b9a14858d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2daa50738ae7444a982e73f0b39c271e" is_pre_election: true
I20260929 03:05:33.565599 10190 raft_consensus.cc:2454] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80e1755b229e41219d14e6b9a14858d0 in term 0.
I20260929 03:05:33.565562 9909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "652c2017ff3f4758928c7d6e68290770" candidate_uuid: "80e1755b229e41219d14e6b9a14858d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" is_pre_election: true
I20260929 03:05:33.565717 9909 raft_consensus.cc:2454] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80e1755b229e41219d14e6b9a14858d0 in term 0.
I20260929 03:05:33.566020 9981 leader_election.cc:302] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2daa50738ae7444a982e73f0b39c271e, 3fedc543ed7248b68b7f4b66b45d55c3, 80e1755b229e41219d14e6b9a14858d0; no voters:
I20260929 03:05:33.566254 10339 raft_consensus.cc:2781] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:33.566306 10339 raft_consensus.cc:482] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:33.566340 10339 raft_consensus.cc:3037] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:33.567351 10339 raft_consensus.cc:504] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.567724 10339 leader_election.cc:288] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [CANDIDATE]: Term 1 election: Requested vote from peers 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229), 2daa50738ae7444a982e73f0b39c271e (127.7.209.67:33525)
I20260929 03:05:33.567996 9909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "652c2017ff3f4758928c7d6e68290770" candidate_uuid: "80e1755b229e41219d14e6b9a14858d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fedc543ed7248b68b7f4b66b45d55c3"
I20260929 03:05:33.568153 9909 raft_consensus.cc:3037] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:33.569123 9909 raft_consensus.cc:2454] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80e1755b229e41219d14e6b9a14858d0 in term 1.
I20260929 03:05:33.569346 10190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "652c2017ff3f4758928c7d6e68290770" candidate_uuid: "80e1755b229e41219d14e6b9a14858d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2daa50738ae7444a982e73f0b39c271e"
I20260929 03:05:33.569384 9981 leader_election.cc:302] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fedc543ed7248b68b7f4b66b45d55c3, 80e1755b229e41219d14e6b9a14858d0; no voters:
I20260929 03:05:33.569461 10190 raft_consensus.cc:3037] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:33.569527 10339 raft_consensus.cc:2781] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:33.569638 10339 raft_consensus.cc:686] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 1 LEADER]: Becoming Leader. State: Replica: 80e1755b229e41219d14e6b9a14858d0, State: Running, Role: LEADER
I20260929 03:05:33.569778 10339 consensus_queue.cc:237] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.570474 10190 raft_consensus.cc:2454] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80e1755b229e41219d14e6b9a14858d0 in term 1.
I20260929 03:05:33.571225 9753 catalog_manager.cc:5495] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66). New cstate: current_term: 1 leader_uuid: "80e1755b229e41219d14e6b9a14858d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } health_report { overall_health: UNKNOWN } } }
I20260929 03:05:33.589062 10367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:33.585986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53686 (local address 127.7.209.66:40031)
0929 03:05:33.586302 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:33.586310 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:33.586322 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:05:33.586363 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:33.586366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:33.586409 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:33.586489 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:33.586715 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:33.587289 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:33.588123 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:33.588351 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:33.588421 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:33.588612 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:33.588688 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:33.588738 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:33.588842 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 03:05:33.588893 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":155,"threads_started":1}
I20260929 03:05:33.596398 9909 raft_consensus.cc:1261] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Refusing update from remote peer 80e1755b229e41219d14e6b9a14858d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:33.596486 10190 raft_consensus.cc:1261] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Refusing update from remote peer 80e1755b229e41219d14e6b9a14858d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:33.596786 10339 consensus_queue.cc:1035] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:33.596958 10326 consensus_queue.cc:1035] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:33.627279 9751 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:33.627468 9751 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:33.629777 9751 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea: Sending DeleteTablet for 3 replicas of tablet 652c2017ff3f4758928c7d6e68290770
I20260929 03:05:33.630362 9890 tablet_service.cc:1514] Processing DeleteTablet for tablet 652c2017ff3f4758928c7d6e68290770 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:33 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:58608
I20260929 03:05:33.630393 10030 tablet_service.cc:1514] Processing DeleteTablet for tablet 652c2017ff3f4758928c7d6e68290770 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:33 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:53666
I20260929 03:05:33.630472 10170 tablet_service.cc:1514] Processing DeleteTablet for tablet 652c2017ff3f4758928c7d6e68290770 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:33 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:51778
I20260929 03:05:33.630585 10347 tablet_replica.cc:330] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0: stopping tablet replica
I20260929 03:05:33.630595 10348 tablet_replica.cc:330] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3: stopping tablet replica
I20260929 03:05:33.630712 10347 raft_consensus.cc:2229] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:05:33.630739 10348 raft_consensus.cc:2229] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:33.630839 10349 tablet_replica.cc:330] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e: stopping tablet replica
I20260929 03:05:33.630865 10348 raft_consensus.cc:2258] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:33.630934 10347 raft_consensus.cc:2258] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:33.630990 10349 raft_consensus.cc:2229] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:33.631114 10349 raft_consensus.cc:2258] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:33.631700 10348 ts_tablet_manager.cc:1905] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:33.631592 9751 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 03:05:33.631860 10347 ts_tablet_manager.cc:1905] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:33.631873 10349 ts_tablet_manager.cc:1905] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 03:05:33.632091 9751 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:33.634979 10349 ts_tablet_manager.cc:1918] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:05:33.635144 10349 log.cc:1196] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/652c2017ff3f4758928c7d6e68290770
I20260929 03:05:33.635190 10347 ts_tablet_manager.cc:1918] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:05:33.635218 10348 ts_tablet_manager.cc:1918] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:05:33.635274 10347 log.cc:1196] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/652c2017ff3f4758928c7d6e68290770
I20260929 03:05:33.635353 10348 log.cc:1196] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-0/wal/wals/652c2017ff3f4758928c7d6e68290770
I20260929 03:05:33.635610 10349 ts_tablet_manager.cc:1939] T 652c2017ff3f4758928c7d6e68290770 P 2daa50738ae7444a982e73f0b39c271e: Deleting consensus metadata
I20260929 03:05:33.635660 10348 ts_tablet_manager.cc:1939] T 652c2017ff3f4758928c7d6e68290770 P 3fedc543ed7248b68b7f4b66b45d55c3: Deleting consensus metadata
I20260929 03:05:33.636435 10347 ts_tablet_manager.cc:1939] T 652c2017ff3f4758928c7d6e68290770 P 80e1755b229e41219d14e6b9a14858d0: Deleting consensus metadata
I20260929 03:05:33.636446 9740 catalog_manager.cc:4841] TS 2daa50738ae7444a982e73f0b39c271e (127.7.209.67:33525): tablet 652c2017ff3f4758928c7d6e68290770 (table test-table [id=8fd0cdde30b54026b7f59a72eb100735]) successfully deleted
I20260929 03:05:33.637166 9740 catalog_manager.cc:4841] TS 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66:40031): tablet 652c2017ff3f4758928c7d6e68290770 (table test-table [id=8fd0cdde30b54026b7f59a72eb100735]) successfully deleted
I20260929 03:05:33.637285 9737 catalog_manager.cc:4841] TS 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229): tablet 652c2017ff3f4758928c7d6e68290770 (table test-table [id=8fd0cdde30b54026b7f59a72eb100735]) successfully deleted
I20260929 03:05:33.638144 9890 tablet_service.cc:1467] Processing CreateTablet for tablet 45abd7e4040d46ab8042dd45937bdc68 (DEFAULT_TABLE table=test-table [id=58f0b1a3f5154957bb69d2bb8a762868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:33.638247 10030 tablet_service.cc:1467] Processing CreateTablet for tablet 45abd7e4040d46ab8042dd45937bdc68 (DEFAULT_TABLE table=test-table [id=58f0b1a3f5154957bb69d2bb8a762868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:33.638278 10170 tablet_service.cc:1467] Processing CreateTablet for tablet 45abd7e4040d46ab8042dd45937bdc68 (DEFAULT_TABLE table=test-table [id=58f0b1a3f5154957bb69d2bb8a762868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:33.638422 9890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45abd7e4040d46ab8042dd45937bdc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:33.638504 10030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45abd7e4040d46ab8042dd45937bdc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:33.638504 10170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45abd7e4040d46ab8042dd45937bdc68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:33.640409 10360 tablet_bootstrap.cc:492] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3: Bootstrap starting.
I20260929 03:05:33.640741 10359 tablet_bootstrap.cc:492] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e: Bootstrap starting.
I20260929 03:05:33.640818 10358 tablet_bootstrap.cc:492] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0: Bootstrap starting.
I20260929 03:05:33.641319 10360 tablet_bootstrap.cc:654] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:33.641599 10359 tablet_bootstrap.cc:654] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:33.641742 10358 tablet_bootstrap.cc:654] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:33.642551 10360 tablet_bootstrap.cc:492] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3: No bootstrap required, opened a new log
I20260929 03:05:33.642642 10360 ts_tablet_manager.cc:1397] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260929 03:05:33.642762 10358 tablet_bootstrap.cc:492] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0: No bootstrap required, opened a new log
I20260929 03:05:33.642760 10359 tablet_bootstrap.cc:492] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e: No bootstrap required, opened a new log
I20260929 03:05:33.642832 10358 ts_tablet_manager.cc:1397] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 03:05:33.642863 10359 ts_tablet_manager.cc:1397] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 03:05:33.643167 10360 raft_consensus.cc:348] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.643237 10358 raft_consensus.cc:348] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.643303 10360 raft_consensus.cc:374] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:33.643250 10359 raft_consensus.cc:348] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.643330 10358 raft_consensus.cc:374] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:33.643337 10360 raft_consensus.cc:729] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fedc543ed7248b68b7f4b66b45d55c3, State: Initialized, Role: FOLLOWER
I20260929 03:05:33.643352 10359 raft_consensus.cc:374] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:33.643381 10359 raft_consensus.cc:729] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2daa50738ae7444a982e73f0b39c271e, State: Initialized, Role: FOLLOWER
I20260929 03:05:33.643368 10358 raft_consensus.cc:729] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80e1755b229e41219d14e6b9a14858d0, State: Initialized, Role: FOLLOWER
I20260929 03:05:33.643491 10360 consensus_queue.cc:260] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.643504 10358 consensus_queue.cc:260] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.643507 10359 consensus_queue.cc:260] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.643767 10358 ts_tablet_manager.cc:1428] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:33.643803 10359 ts_tablet_manager.cc:1428] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:33.643879 10360 ts_tablet_manager.cc:1428] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260929 03:05:33.669842 10265 raft_consensus.cc:482] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:33.669993 10265 raft_consensus.cc:504] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.670513 10265 leader_election.cc:288] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229), 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66:40031)
I20260929 03:05:33.670964 9909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45abd7e4040d46ab8042dd45937bdc68" candidate_uuid: "2daa50738ae7444a982e73f0b39c271e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" is_pre_election: true
I20260929 03:05:33.671000 10049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45abd7e4040d46ab8042dd45937bdc68" candidate_uuid: "2daa50738ae7444a982e73f0b39c271e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80e1755b229e41219d14e6b9a14858d0" is_pre_election: true
I20260929 03:05:33.671187 10049 raft_consensus.cc:2454] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2daa50738ae7444a982e73f0b39c271e in term 0.
I20260929 03:05:33.671180 9909 raft_consensus.cc:2454] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2daa50738ae7444a982e73f0b39c271e in term 0.
I20260929 03:05:33.671583 10124 leader_election.cc:302] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2daa50738ae7444a982e73f0b39c271e, 3fedc543ed7248b68b7f4b66b45d55c3, 80e1755b229e41219d14e6b9a14858d0; no voters:
I20260929 03:05:33.671823 10265 raft_consensus.cc:2781] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:33.671886 10265 raft_consensus.cc:482] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:33.671914 10265 raft_consensus.cc:3037] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:33.673314 10265 raft_consensus.cc:504] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.673740 10265 leader_election.cc:288] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [CANDIDATE]: Term 1 election: Requested vote from peers 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229), 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66:40031)
I20260929 03:05:33.674155 10049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45abd7e4040d46ab8042dd45937bdc68" candidate_uuid: "2daa50738ae7444a982e73f0b39c271e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80e1755b229e41219d14e6b9a14858d0"
I20260929 03:05:33.674144 9909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45abd7e4040d46ab8042dd45937bdc68" candidate_uuid: "2daa50738ae7444a982e73f0b39c271e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fedc543ed7248b68b7f4b66b45d55c3"
I20260929 03:05:33.674301 9909 raft_consensus.cc:3037] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:33.674302 10049 raft_consensus.cc:3037] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:33.675810 10049 raft_consensus.cc:2454] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2daa50738ae7444a982e73f0b39c271e in term 1.
I20260929 03:05:33.675812 9909 raft_consensus.cc:2454] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2daa50738ae7444a982e73f0b39c271e in term 1.
I20260929 03:05:33.676131 10121 leader_election.cc:302] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2daa50738ae7444a982e73f0b39c271e, 3fedc543ed7248b68b7f4b66b45d55c3; no voters:
I20260929 03:05:33.676298 10265 raft_consensus.cc:2781] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:33.676389 10265 raft_consensus.cc:686] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 1 LEADER]: Becoming Leader. State: Replica: 2daa50738ae7444a982e73f0b39c271e, State: Running, Role: LEADER
I20260929 03:05:33.676534 10265 consensus_queue.cc:237] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } }
I20260929 03:05:33.678438 9753 catalog_manager.cc:5495] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2daa50738ae7444a982e73f0b39c271e (127.7.209.67). New cstate: current_term: 1 leader_uuid: "2daa50738ae7444a982e73f0b39c271e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2daa50738ae7444a982e73f0b39c271e" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33525 } health_report { overall_health: HEALTHY } } }
I20260929 03:05:33.701571 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:33.698205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51808 (local address 127.7.209.67:33525)
0929 03:05:33.698534 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:33.698542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:33.698553 (+ 11us) server_negotiation.cc:407] Connection header received
0929 03:05:33.698592 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:33.698594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:33.698657 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:33.698736 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:33.698968 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:33.699519 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:33.700621 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:33.700855 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:33.700930 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:33.701166 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:33.701243 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:33.701289 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:33.701366 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 03:05:33.701420 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":171,"threads_started":1}
I20260929 03:05:33.714407 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:33.708031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:49745 (local address 127.7.209.67:33525)
0929 03:05:33.708142 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:33.708145 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:33.711408 (+ 3263us) server_negotiation.cc:407] Connection header received
0929 03:05:33.711527 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:33.711530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:33.711569 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:33.711646 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:33.711918 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:33.712496 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:33.713358 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:33.713568 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:33.713695 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:33.713916 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:33.713993 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:33.714050 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:33.714131 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 03:05:33.714222 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":32}
I20260929 03:05:33.714407 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:33.707898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:33525 (local address 127.7.209.67:49745)
0929 03:05:33.711300 (+ 3402us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:33.711313 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:33.711432 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:33.711640 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:33.711643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:33.711651 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 03:05:33.711827 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:33.711833 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:33.712601 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:33.712604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:33.713250 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:33.713259 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:33.713706 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:33.713709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:33.713795 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:33.713817 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 03:05:33.714058 (+ 241us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 03:05:33.714064 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 03:05:33.714125 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 03:05:33.714242 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3327,"mutex_wait_us":30,"thread_start_us":149,"threads_started":1}
I20260929 03:05:33.720546 9909 raft_consensus.cc:1261] T 45abd7e4040d46ab8042dd45937bdc68 P 3fedc543ed7248b68b7f4b66b45d55c3 [term 1 FOLLOWER]: Refusing update from remote peer 2daa50738ae7444a982e73f0b39c271e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:33.720544 10049 raft_consensus.cc:1261] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Refusing update from remote peer 2daa50738ae7444a982e73f0b39c271e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:33.720933 10329 consensus_queue.cc:1035] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [LEADER]: Connected to new peer: Peer: permanent_uuid: "80e1755b229e41219d14e6b9a14858d0" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:33.721065 10265 consensus_queue.cc:1035] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fedc543ed7248b68b7f4b66b45d55c3" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 03:05:33.734944 10327 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 03:05:35.293217 9750 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:35.293413 9750 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:35.295967 9750 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c4cf66131cb462a92ff2c423e14c1ea: Sending DeleteTablet for 3 replicas of tablet 45abd7e4040d46ab8042dd45937bdc68
I20260929 03:05:35.296501 9890 tablet_service.cc:1514] Processing DeleteTablet for tablet 45abd7e4040d46ab8042dd45937bdc68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:35 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:58608
I20260929 03:05:35.296582 10169 tablet_service.cc:1514] Processing DeleteTablet for tablet 45abd7e4040d46ab8042dd45937bdc68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:35 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:51778
I20260929 03:05:35.296562 10030 tablet_service.cc:1514] Processing DeleteTablet for tablet 45abd7e4040d46ab8042dd45937bdc68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:35 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:53666
I20260929 03:05:35.296932 10404 tablet_replica.cc:330] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0: stopping tablet replica
I20260929 03:05:35.296931 10405 tablet_replica.cc:330] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e: stopping tablet replica
I20260929 03:05:35.297025 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 9825
I20260929 03:05:35.297075 10405 raft_consensus.cc:2229] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:05:35.297102 10404 raft_consensus.cc:2229] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:35.298098 10404 raft_consensus.cc:2258] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:35.298817 10404 ts_tablet_manager.cc:1905] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:35.300463 10405 raft_consensus.cc:2258] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:35.301065 10405 ts_tablet_manager.cc:1905] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:35.302117 10404 ts_tablet_manager.cc:1918] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 03:05:35.302198 10404 log.cc:1196] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/45abd7e4040d46ab8042dd45937bdc68
I20260929 03:05:35.302541 10404 ts_tablet_manager.cc:1939] T 45abd7e4040d46ab8042dd45937bdc68 P 80e1755b229e41219d14e6b9a14858d0: Deleting consensus metadata
I20260929 03:05:35.303350 9740 catalog_manager.cc:4841] TS 80e1755b229e41219d14e6b9a14858d0 (127.7.209.66:40031): tablet 45abd7e4040d46ab8042dd45937bdc68 (table test-table [id=58f0b1a3f5154957bb69d2bb8a762868]) successfully deleted
I20260929 03:05:35.303817 10405 ts_tablet_manager.cc:1918] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 03:05:35.303881 10405 log.cc:1196] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/45abd7e4040d46ab8042dd45937bdc68
I20260929 03:05:35.304116 10405 ts_tablet_manager.cc:1939] T 45abd7e4040d46ab8042dd45937bdc68 P 2daa50738ae7444a982e73f0b39c271e: Deleting consensus metadata
I20260929 03:05:35.304752 9740 catalog_manager.cc:4841] TS 2daa50738ae7444a982e73f0b39c271e (127.7.209.67:33525): tablet 45abd7e4040d46ab8042dd45937bdc68 (table test-table [id=58f0b1a3f5154957bb69d2bb8a762868]) successfully deleted
W20260929 03:05:35.305776 9737 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:35229 (error 108)
I20260929 03:05:35.306900 10407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:35.306456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:35229 (local address 127.0.0.1:58622)
0929 03:05:35.306680 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:35.306785 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:35229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
I20260929 03:05:35.306982 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 9965
W20260929 03:05:35.307036 9737 catalog_manager.cc:4584] TS 3fedc543ed7248b68b7f4b66b45d55c3 (127.7.209.65:35229): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 45abd7e4040d46ab8042dd45937bdc68: Network error: Client connection negotiation failed: client connection to 127.7.209.65:35229: connect: Connection refused (error 111)
I20260929 03:05:35.313891 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 10105
I20260929 03:05:35.320684 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 9721
2026-09-29T03:05:35Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6495 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:05:35 dist-test-slave-bdps krb5kdc[10410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:05:35 dist-test-slave-bdps krb5kdc[10410](info): set up 2 sockets
Sep 29 03:05:35 dist-test-slave-bdps krb5kdc[10410](info): commencing operation
krb5kdc: starting...
W20260929 03:05:36.277933 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:05:36 dist-test-slave-bdps krb5kdc[10410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:05:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:05:36.404791 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:41689
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:41689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:36.529338 10423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:36.529701 10423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:36.529765 10423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:36.533825 10423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:36.533890 10423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:36.533917 10423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:36.533938 10423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:36.533958 10423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:36.539340 10423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:41689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:41689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:36.540549 10423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:36.541808 10423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:36.550882 10429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:36.550900 10431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:36.550920 10428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:36.551236 10423 server_base.cc:1034] running on GCE node
I20260929 03:05:36.551834 10423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:36.553143 10423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:36.554303 10423 hybrid_clock.cc:648] HybridClock initialized: now 1790651136554285 us; error 34 us; skew 500 ppm
Sep 29 03:05:36 dist-test-slave-bdps krb5kdc[10410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651136, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:36.557426 10423 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:05:36.558511 10423 webserver.cc:460] Webserver started at http://127.7.209.126:45653/ using document root <none> and password file <none>
I20260929 03:05:36.559177 10423 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:36.559250 10423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:36.559490 10423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:36.562016 10423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "0ff8519f7e4647ff91944d4d7dd27602"
format_stamp: "Formatted at 2026-09-29 03:05:36 on dist-test-slave-bdps"
server_key: "6f7eccd43e3636e3221ba29b025a22a4"
server_key_iv: "1d0a34a05ed81d79c7e280197405edb3"
server_key_version: "encryptionkey@0"
I20260929 03:05:36.562664 10423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "0ff8519f7e4647ff91944d4d7dd27602"
format_stamp: "Formatted at 2026-09-29 03:05:36 on dist-test-slave-bdps"
server_key: "6f7eccd43e3636e3221ba29b025a22a4"
server_key_iv: "1d0a34a05ed81d79c7e280197405edb3"
server_key_version: "encryptionkey@0"
I20260929 03:05:36.566936 10423 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260929 03:05:36.569563 10438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:36.570962 10423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:36.571082 10423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "0ff8519f7e4647ff91944d4d7dd27602"
format_stamp: "Formatted at 2026-09-29 03:05:36 on dist-test-slave-bdps"
server_key: "6f7eccd43e3636e3221ba29b025a22a4"
server_key_iv: "1d0a34a05ed81d79c7e280197405edb3"
server_key_version: "encryptionkey@0"
I20260929 03:05:36.571192 10423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:36.587682 10423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:36.591300 10423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:36.591507 10423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:36.599769 10423 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:41689
I20260929 03:05:36.599803 10490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:41689 every 8 connection(s)
I20260929 03:05:36.600951 10423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:36.602167 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 10423
I20260929 03:05:36.602301 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:36.602658 8005 external_mini_cluster.cc:1428] Setting key 4554e6fe141c1cc9083188b12870088e
I20260929 03:05:36.605088 10491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:36.612023 10491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602: Bootstrap starting.
Sep 29 03:05:36 dist-test-slave-bdps krb5kdc[10410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:36.614976 10491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:36.615803 10491 log.cc:824] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:36.618295 10491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602: No bootstrap required, opened a new log
I20260929 03:05:36.618458 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:36.604457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54168 (local address 127.7.209.126:41689)
0929 03:05:36.604951 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:36.604969 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:36.605012 (+ 43us) server_negotiation.cc:407] Connection header received
0929 03:05:36.605849 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:36.605872 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:36.606286 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:36.606677 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:36.607548 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:36.608726 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:36.609799 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:36.610341 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:36.610502 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:36.613322 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:36.613327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:36.613342 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:36.613371 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:36.615134 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:36.615573 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:36.615579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:36.615584 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:36.615708 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:36.616014 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:36.616017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:36.616019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:36.616532 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:36.616756 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:36.616999 (+ 243us) server_negotiation.cc:299] Negotiation successful
0929 03:05:36.617340 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":170,"threads_started":1}
I20260929 03:05:36.622460 10491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff8519f7e4647ff91944d4d7dd27602" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41689 } }
I20260929 03:05:36.622709 10491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:36.622752 10491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ff8519f7e4647ff91944d4d7dd27602, State: Initialized, Role: FOLLOWER
I20260929 03:05:36.623169 10491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff8519f7e4647ff91944d4d7dd27602" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41689 } }
I20260929 03:05:36.623291 10491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:36.623337 10491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:36.623410 10491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:36.624306 10491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff8519f7e4647ff91944d4d7dd27602" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41689 } }
I20260929 03:05:36.624673 10491 leader_election.cc:302] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ff8519f7e4647ff91944d4d7dd27602; no voters:
I20260929 03:05:36.625033 10491 leader_election.cc:288] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:36.625144 10496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:36.625350 10496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [term 1 LEADER]: Becoming Leader. State: Replica: 0ff8519f7e4647ff91944d4d7dd27602, State: Running, Role: LEADER
I20260929 03:05:36.625769 10496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff8519f7e4647ff91944d4d7dd27602" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41689 } }
I20260929 03:05:36.626413 10491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:36.627445 10497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ff8519f7e4647ff91944d4d7dd27602" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff8519f7e4647ff91944d4d7dd27602" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41689 } } }
I20260929 03:05:36.627589 10497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:36.627987 10505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:36.627955 10498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ff8519f7e4647ff91944d4d7dd27602. Latest consensus state: current_term: 1 leader_uuid: "0ff8519f7e4647ff91944d4d7dd27602" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff8519f7e4647ff91944d4d7dd27602" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41689 } } }
I20260929 03:05:36.628147 10498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:36.631505 10505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:36.637951 10505 catalog_manager.cc:1347] Generated new cluster ID: cd214d05d34d40fa8e13f4764965114b
I20260929 03:05:36.638038 10505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:36.655920 10505 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:36.657348 10505 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:36.662999 10505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ff8519f7e4647ff91944d4d7dd27602: Generated new TSK 0
I20260929 03:05:36.663924 10505 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:05:36.702145 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:41689
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:36.826004 10517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:36.826288 10517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:36.826330 10517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:36.830435 10517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:36.830494 10517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:36.830549 10517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:36.835886 10517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:41689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:36.837146 10517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:36.838634 10517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:36.848839 10523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:36.848840 10522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:36.848904 10525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:36.849119 10517 server_base.cc:1034] running on GCE node
I20260929 03:05:36.849545 10517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:36.850206 10517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:36.851367 10517 hybrid_clock.cc:648] HybridClock initialized: now 1790651136851344 us; error 35 us; skew 500 ppm
Sep 29 03:05:36 dist-test-slave-bdps krb5kdc[10410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651136, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:36.854208 10517 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:05:36.855211 10517 webserver.cc:460] Webserver started at http://127.7.209.65:36779/ using document root <none> and password file <none>
I20260929 03:05:36.855805 10517 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:36.855871 10517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:36.856097 10517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:36.858469 10517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "bb08afeb091a43b7ab56f4ef9b5b55ea"
format_stamp: "Formatted at 2026-09-29 03:05:36 on dist-test-slave-bdps"
server_key: "e67089ca635c40719d0c8e0881d54f01"
server_key_iv: "71a85df2029ca1d0d8db66515539b505"
server_key_version: "encryptionkey@0"
I20260929 03:05:36.859076 10517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "bb08afeb091a43b7ab56f4ef9b5b55ea"
format_stamp: "Formatted at 2026-09-29 03:05:36 on dist-test-slave-bdps"
server_key: "e67089ca635c40719d0c8e0881d54f01"
server_key_iv: "71a85df2029ca1d0d8db66515539b505"
server_key_version: "encryptionkey@0"
I20260929 03:05:36.863274 10517 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:05:36.865980 10532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:36.867323 10517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:05:36.867434 10517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "bb08afeb091a43b7ab56f4ef9b5b55ea"
format_stamp: "Formatted at 2026-09-29 03:05:36 on dist-test-slave-bdps"
server_key: "e67089ca635c40719d0c8e0881d54f01"
server_key_iv: "71a85df2029ca1d0d8db66515539b505"
server_key_version: "encryptionkey@0"
I20260929 03:05:36.867535 10517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:36.883766 10517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:36.887180 10517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:36.887383 10517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:36.888087 10517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:36.889035 10517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:36.889084 10517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:36.889151 10517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:36.889176 10517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:36.899119 10517 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:37321
I20260929 03:05:36.899170 10645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:37321 every 8 connection(s)
I20260929 03:05:36.900172 10517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:36.909065 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 10517
I20260929 03:05:36.909190 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:05:36.909499 8005 external_mini_cluster.cc:1428] Setting key cc5aa3e049766a5bb726a422abff652b
Sep 29 03:05:36 dist-test-slave-bdps krb5kdc[10410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651136, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:36.915088 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:36.902281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:44309 (local address 127.7.209.126:41689)
0929 03:05:36.902460 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:36.902465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:36.903617 (+ 1152us) server_negotiation.cc:407] Connection header received
0929 03:05:36.904582 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:36.904585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:36.904639 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:36.904706 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:36.906064 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:36.906663 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:36.907819 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:36.908059 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:36.908131 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:36.910932 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:36.910936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:36.910939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:36.910963 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:36.912582 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:36.913137 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:36.913148 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:36.913151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:36.913256 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:36.913637 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:36.913640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:36.913641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:36.913935 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:36.914018 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:36.914779 (+ 761us) server_negotiation.cc:299] Negotiation successful
0929 03:05:36.914930 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 03:05:36.916572 10648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:36.902748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:41689 (local address 127.7.209.65:44309)
0929 03:05:36.903452 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:36.903498 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:36.904368 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:36.904895 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:36.904903 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:36.905306 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:36.905922 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:36.905938 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:36.906761 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:36.906763 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:36.907726 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:36.907735 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:36.908181 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:36.908184 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:36.908337 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:36.908960 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:36.908976 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:36.910760 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:36.912711 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:36.912718 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:36.912737 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:36.913045 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:36.913351 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:36.913353 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:36.913355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:36.913528 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:36.914043 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:36.914053 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:36.914501 (+ 448us) client_negotiation.cc:769] Sending connection context
0929 03:05:36.914762 (+ 261us) client_negotiation.cc:240] Negotiation successful
0929 03:05:36.915239 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":124,"threads_started":1}
I20260929 03:05:36.918074 10646 heartbeater.cc:347] Connected to a master server at 127.7.209.126:41689
I20260929 03:05:36.918385 10646 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:36.918995 10646 heartbeater.cc:511] Master 127.7.209.126:41689 requested a full tablet report, sending...
I20260929 03:05:36.920825 10455 ts_manager.cc:194] Registered new tserver with Master: bb08afeb091a43b7ab56f4ef9b5b55ea (127.7.209.65:37321)
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:05:36.922260 10455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:44309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:05:36.938825 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:41689
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:37.084619 10651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:37.084970 10651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:37.085026 10651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:37.089387 10651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:37.089486 10651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:37.089567 10651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:37.095448 10651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:41689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:37.096882 10651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:37.098263 10651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:37.109789 10657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:37.109804 10656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:37.109823 10659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:37.110715 10651 server_base.cc:1034] running on GCE node
I20260929 03:05:37.111233 10651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:37.112035 10651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:37.113207 10651 hybrid_clock.cc:648] HybridClock initialized: now 1790651137113183 us; error 40 us; skew 500 ppm
Sep 29 03:05:37 dist-test-slave-bdps krb5kdc[10410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:37.116460 10651 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:05:37.117859 10651 webserver.cc:460] Webserver started at http://127.7.209.66:41553/ using document root <none> and password file <none>
I20260929 03:05:37.118528 10651 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:37.118611 10651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:37.118856 10651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:37.121311 10651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "c204eb6058894e759db33ff58bfb78b8"
format_stamp: "Formatted at 2026-09-29 03:05:37 on dist-test-slave-bdps"
server_key: "abdb6626f7fbc9f781c137df3461d78c"
server_key_iv: "d44b3b1f69b12c3474c0ad43fb24c8f9"
server_key_version: "encryptionkey@0"
I20260929 03:05:37.121986 10651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "c204eb6058894e759db33ff58bfb78b8"
format_stamp: "Formatted at 2026-09-29 03:05:37 on dist-test-slave-bdps"
server_key: "abdb6626f7fbc9f781c137df3461d78c"
server_key_iv: "d44b3b1f69b12c3474c0ad43fb24c8f9"
server_key_version: "encryptionkey@0"
I20260929 03:05:37.126281 10651 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:05:37.129021 10666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:37.130439 10651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:37.130566 10651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "c204eb6058894e759db33ff58bfb78b8"
format_stamp: "Formatted at 2026-09-29 03:05:37 on dist-test-slave-bdps"
server_key: "abdb6626f7fbc9f781c137df3461d78c"
server_key_iv: "d44b3b1f69b12c3474c0ad43fb24c8f9"
server_key_version: "encryptionkey@0"
I20260929 03:05:37.130677 10651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:37.146927 10651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:37.149720 10651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:37.149886 10651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:37.150470 10651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:37.151392 10651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:37.151439 10651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:37.151494 10651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:37.151518 10651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:37.161441 10651 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:38951
I20260929 03:05:37.161464 10779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:38951 every 8 connection(s)
I20260929 03:05:37.162609 10651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:37.166327 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 10651
I20260929 03:05:37.166448 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:05:37.166775 8005 external_mini_cluster.cc:1428] Setting key 81f14c0cddd1e3ddabeb1df51e4bfda6
Sep 29 03:05:37 dist-test-slave-bdps krb5kdc[10410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:05:37.178361 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:37.164783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:39783 (local address 127.7.209.126:41689)
0929 03:05:37.164939 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:37.164943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:37.166211 (+ 1268us) server_negotiation.cc:407] Connection header received
0929 03:05:37.167470 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:37.167473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:37.167540 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:37.167633 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:37.169204 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:37.169795 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:37.171306 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:37.171545 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:37.171640 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:37.174565 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:37.174568 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:37.174571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:37.174596 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:37.176123 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:37.176735 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:37.176738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:37.176739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:37.176818 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:37.177187 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:37.177189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:37.177191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:37.177357 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:37.177429 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:37.178096 (+ 667us) server_negotiation.cc:299] Negotiation successful
0929 03:05:37.178218 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 03:05:37.179446 10782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:37.165264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:41689 (local address 127.7.209.66:39783)
0929 03:05:37.166012 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:37.166078 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:37.167241 (+ 1163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:37.167854 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:37.167862 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:37.168278 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:37.169053 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:37.169075 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:37.169922 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:37.169925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:37.171190 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:37.171200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:37.171695 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:37.171700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:37.171867 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:37.172519 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:37.172537 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:37.174397 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:37.176244 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:37.176251 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:37.176269 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:37.176641 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:37.176896 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:37.176900 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:37.176902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:37.177085 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:37.177449 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:37.177457 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:37.177823 (+ 366us) client_negotiation.cc:769] Sending connection context
0929 03:05:37.178062 (+ 239us) client_negotiation.cc:240] Negotiation successful
0929 03:05:37.178400 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":124,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
I20260929 03:05:37.180712 10780 heartbeater.cc:347] Connected to a master server at 127.7.209.126:41689
I20260929 03:05:37.180975 10780 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:37.181552 10780 heartbeater.cc:511] Master 127.7.209.126:41689 requested a full tablet report, sending...
I20260929 03:05:37.182756 10455 ts_manager.cc:194] Registered new tserver with Master: c204eb6058894e759db33ff58bfb78b8 (127.7.209.66:38951)
I20260929 03:05:37.183588 10455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:39783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:05:37.194648 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:41689
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:37.321056 10785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:37.321375 10785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:37.321425 10785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:37.325556 10785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:37.325646 10785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:37.325732 10785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:37.331248 10785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:41689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:37.332456 10785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:37.333727 10785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:37.344340 10790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:37.344429 10791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:37.344404 10785 server_base.cc:1034] running on GCE node
W20260929 03:05:37.344734 10793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:37.345129 10785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:37.345853 10785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:37.347018 10785 hybrid_clock.cc:648] HybridClock initialized: now 1790651137346993 us; error 39 us; skew 500 ppm
Sep 29 03:05:37 dist-test-slave-bdps krb5kdc[10410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:37.350092 10785 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:05:37.351019 10785 webserver.cc:460] Webserver started at http://127.7.209.67:37495/ using document root <none> and password file <none>
I20260929 03:05:37.351642 10785 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:37.351706 10785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:37.351908 10785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:37.354413 10785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "dd0629aef8f04788a00e6fe19a49348a"
format_stamp: "Formatted at 2026-09-29 03:05:37 on dist-test-slave-bdps"
server_key: "269fa30177fc4c2361e9d3fb97f6d5c6"
server_key_iv: "6eaf5a1cb1a8546dac257ae6c3054cf4"
server_key_version: "encryptionkey@0"
I20260929 03:05:37.355038 10785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "dd0629aef8f04788a00e6fe19a49348a"
format_stamp: "Formatted at 2026-09-29 03:05:37 on dist-test-slave-bdps"
server_key: "269fa30177fc4c2361e9d3fb97f6d5c6"
server_key_iv: "6eaf5a1cb1a8546dac257ae6c3054cf4"
server_key_version: "encryptionkey@0"
I20260929 03:05:37.359395 10785 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:05:37.362288 10800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:37.363778 10785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:37.363898 10785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "dd0629aef8f04788a00e6fe19a49348a"
format_stamp: "Formatted at 2026-09-29 03:05:37 on dist-test-slave-bdps"
server_key: "269fa30177fc4c2361e9d3fb97f6d5c6"
server_key_iv: "6eaf5a1cb1a8546dac257ae6c3054cf4"
server_key_version: "encryptionkey@0"
I20260929 03:05:37.363999 10785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:37.394896 10785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:37.398808 10785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:37.398993 10785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:37.399679 10785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:37.400640 10785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:37.400701 10785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:37.400746 10785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:37.400766 10785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:37.410422 10785 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:45109
I20260929 03:05:37.410473 10913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:45109 every 8 connection(s)
I20260929 03:05:37.411453 10785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:37.411841 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 10785
I20260929 03:05:37.411963 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:05:37.412351 8005 external_mini_cluster.cc:1428] Setting key 0cb5892b5dd666094bc3f9d1bddcffec
Sep 29 03:05:37 dist-test-slave-bdps krb5kdc[10410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651137, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:37.425401 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:37.413690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:36435 (local address 127.7.209.126:41689)
0929 03:05:37.413847 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:37.413851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:37.415213 (+ 1362us) server_negotiation.cc:407] Connection header received
0929 03:05:37.416147 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:37.416150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:37.416201 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:37.416273 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:37.417538 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:37.418183 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:37.419354 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:37.419599 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:37.419715 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:37.422285 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:37.422295 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:37.422297 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:37.422322 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:37.423519 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:37.423979 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:37.423982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:37.423983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:37.424053 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:37.424396 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:37.424398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:37.424399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:37.424576 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:37.424646 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:37.425136 (+ 490us) server_negotiation.cc:299] Negotiation successful
0929 03:05:37.425244 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 03:05:37.426617 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:37.414226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:41689 (local address 127.7.209.67:36435)
0929 03:05:37.415052 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:37.415101 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:37.415954 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:37.416430 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:37.416437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:37.416825 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:37.417404 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:37.417419 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:37.418274 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:37.418277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:37.419227 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:37.419235 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:37.419746 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:37.419749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:37.419895 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:37.420461 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:37.420478 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:37.422115 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:37.423619 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:37.423623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:37.423637 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:37.423894 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:37.424114 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:37.424116 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:37.424117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:37.424325 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:37.424649 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:37.424654 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:37.424909 (+ 255us) client_negotiation.cc:769] Sending connection context
0929 03:05:37.425113 (+ 204us) client_negotiation.cc:240] Negotiation successful
0929 03:05:37.425455 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":161,"threads_started":1}
I20260929 03:05:37.427728 10914 heartbeater.cc:347] Connected to a master server at 127.7.209.126:41689
I20260929 03:05:37.427995 10914 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:37.428613 10914 heartbeater.cc:511] Master 127.7.209.126:41689 requested a full tablet report, sending...
I20260929 03:05:37.429745 10455 ts_manager.cc:194] Registered new tserver with Master: dd0629aef8f04788a00e6fe19a49348a (127.7.209.67:45109)
I20260929 03:05:37.430382 10455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:36435
I20260929 03:05:37.439172 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 03:05:37.455595 10494 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:05:37.453895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54428 (local address 127.7.209.126:41689)
0929 03:05:37.454039 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:37.454044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:37.454227 (+ 183us) server_negotiation.cc:407] Connection header received
0929 03:05:37.454369 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:37.454372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:37.454424 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:37.454509 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:37.455401 (+ 892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54428: BlockingRecv error: recv got EOF from 127.0.0.1:54428 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 03:05:37.456909 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 10517
I20260929 03:05:37.462970 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 10651
I20260929 03:05:37.468564 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 10785
I20260929 03:05:37.473877 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 10423
2026-09-29T03:05:37Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2152 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:05:37 dist-test-slave-bdps krb5kdc[10926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:05:37 dist-test-slave-bdps krb5kdc[10926](info): set up 2 sockets
Sep 29 03:05:37 dist-test-slave-bdps krb5kdc[10926](info): commencing operation
krb5kdc: starting...
W20260929 03:05:38.434034 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:05:38 dist-test-slave-bdps krb5kdc[10926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:05:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:05:38.560822 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:40431
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:45979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:40431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:38.689065 10939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:38.689390 10939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:38.689452 10939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:38.693567 10939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:38.693688 10939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:38.693722 10939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:38.693743 10939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:38.693765 10939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:38.699294 10939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:45979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:40431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:40431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:38.700579 10939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:38.701893 10939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:38.711989 10945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:38.711979 10947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:38.711983 10944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:38.712620 10939 server_base.cc:1034] running on GCE node
I20260929 03:05:38.713405 10939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:38.715040 10939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:38.716216 10939 hybrid_clock.cc:648] HybridClock initialized: now 1790651138716198 us; error 43 us; skew 500 ppm
Sep 29 03:05:38 dist-test-slave-bdps krb5kdc[10926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:38.722944 10939 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:05:38.724124 10939 webserver.cc:460] Webserver started at http://127.7.209.126:32885/ using document root <none> and password file <none>
I20260929 03:05:38.724784 10939 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:38.724848 10939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:38.725054 10939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:38.727726 10939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "191899f86ebd47ec89c0ad091df14e58"
format_stamp: "Formatted at 2026-09-29 03:05:38 on dist-test-slave-bdps"
server_key: "e577bbc9536724e131619913bfcd4ef9"
server_key_iv: "f15830dc5e033be86fde7aa7743d052f"
server_key_version: "encryptionkey@0"
I20260929 03:05:38.728482 10939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "191899f86ebd47ec89c0ad091df14e58"
format_stamp: "Formatted at 2026-09-29 03:05:38 on dist-test-slave-bdps"
server_key: "e577bbc9536724e131619913bfcd4ef9"
server_key_iv: "f15830dc5e033be86fde7aa7743d052f"
server_key_version: "encryptionkey@0"
I20260929 03:05:38.732607 10939 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260929 03:05:38.735183 10954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:38.736515 10939 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 03:05:38.736655 10939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "191899f86ebd47ec89c0ad091df14e58"
format_stamp: "Formatted at 2026-09-29 03:05:38 on dist-test-slave-bdps"
server_key: "e577bbc9536724e131619913bfcd4ef9"
server_key_iv: "f15830dc5e033be86fde7aa7743d052f"
server_key_version: "encryptionkey@0"
I20260929 03:05:38.736747 10939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:38.747063 10939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:38.749847 10939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:38.750018 10939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:38.758011 10939 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:40431
I20260929 03:05:38.758320 11006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:40431 every 8 connection(s)
I20260929 03:05:38.759200 10939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:38.762669 11007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:38.769243 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 10939
I20260929 03:05:38.769368 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:38.769337 11007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58: Bootstrap starting.
I20260929 03:05:38.769713 8005 external_mini_cluster.cc:1428] Setting key cf5d91e3794d0ecb1b4bb33995e764d3
I20260929 03:05:38.772312 11007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:38.773247 11007 log.cc:824] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:38.775528 11007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58: No bootstrap required, opened a new log
Sep 29 03:05:38 dist-test-slave-bdps krb5kdc[10926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:38.780170 11007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191899f86ebd47ec89c0ad091df14e58" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40431 } }
I20260929 03:05:38.780416 11007 raft_consensus.cc:374] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:38.780464 11007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 191899f86ebd47ec89c0ad091df14e58, State: Initialized, Role: FOLLOWER
I20260929 03:05:38.780977 11007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191899f86ebd47ec89c0ad091df14e58" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40431 } }
I20260929 03:05:38.781108 11007 raft_consensus.cc:388] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:38.781203 11007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:38.781307 11007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:38.782294 11007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191899f86ebd47ec89c0ad091df14e58" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40431 } }
I20260929 03:05:38.782697 11007 leader_election.cc:302] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 191899f86ebd47ec89c0ad091df14e58; no voters:
I20260929 03:05:38.783082 11007 leader_election.cc:288] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:38.783216 11012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:38.783422 11012 raft_consensus.cc:686] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [term 1 LEADER]: Becoming Leader. State: Replica: 191899f86ebd47ec89c0ad091df14e58, State: Running, Role: LEADER
I20260929 03:05:38.783834 11012 consensus_queue.cc:237] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191899f86ebd47ec89c0ad091df14e58" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40431 } }
I20260929 03:05:38.784552 11007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:38.785320 11012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 191899f86ebd47ec89c0ad091df14e58. Latest consensus state: current_term: 1 leader_uuid: "191899f86ebd47ec89c0ad091df14e58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191899f86ebd47ec89c0ad091df14e58" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40431 } } }
I20260929 03:05:38.785516 11010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:38.771142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.7.209.126:40431)
0929 03:05:38.771604 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:38.771622 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:38.771667 (+ 45us) server_negotiation.cc:407] Connection header received
0929 03:05:38.772394 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:38.772424 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:38.772850 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:38.773205 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:38.774210 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:38.775288 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:38.776488 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:38.777002 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:38.777143 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:38.780162 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:38.780168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:38.780183 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:38.780212 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:38.782010 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:38.782490 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:38.782497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:38.782503 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:38.782652 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:38.782933 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:38.782936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:38.782937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:38.783547 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:38.783789 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:38.784044 (+ 255us) server_negotiation.cc:299] Negotiation successful
0929 03:05:38.784376 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":134,"threads_started":1}
I20260929 03:05:38.785528 11012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:38.785856 11013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "191899f86ebd47ec89c0ad091df14e58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "191899f86ebd47ec89c0ad091df14e58" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40431 } } }
I20260929 03:05:38.785945 11013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:38.786309 11021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:38.789675 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:05:38.790174 11021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:38.797739 11021 catalog_manager.cc:1347] Generated new cluster ID: 565a75f73c4c48dba58716361cdce2d3
I20260929 03:05:38.797845 11021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:38.818614 11010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:38.812336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44586 (local address 127.7.209.126:40431)
0929 03:05:38.812485 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:38.812491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:38.812808 (+ 317us) server_negotiation.cc:407] Connection header received
0929 03:05:38.812948 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:38.812952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:38.813011 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:38.813117 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:38.813782 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:38.814639 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:38.815821 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:38.816217 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:38.816317 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:38.817297 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:38.817301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:38.817304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:38.818096 (+ 792us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:38.818162 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:38.818184 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:38.818286 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:38.818358 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 03:05:38.818427 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 03:05:38.823845 11021 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:38.825340 11021 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:38.837085 11021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 191899f86ebd47ec89c0ad091df14e58: Generated new TSK 0
I20260929 03:05:38.838343 11021 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:05:38.849536 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 10939
2026-09-29T03:05:38Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1374 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:05:38 dist-test-slave-bdps krb5kdc[11040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:05:38 dist-test-slave-bdps krb5kdc[11040](info): set up 2 sockets
Sep 29 03:05:38 dist-test-slave-bdps krb5kdc[11040](info): commencing operation
krb5kdc: starting...
W20260929 03:05:39.798005 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:05:39 dist-test-slave-bdps krb5kdc[11040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:05:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:05:39.928737 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:35239
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:35239
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:40.054761 11053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:40.055116 11053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:40.055177 11053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:40.059264 11053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:40.059337 11053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:40.059365 11053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:40.059386 11053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:40.059405 11053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:40.065157 11053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:35239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:35239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:40.066607 11053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:40.067965 11053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:40.077823 11061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:40.077910 11059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:40.078068 11053 server_base.cc:1034] running on GCE node
W20260929 03:05:40.077824 11058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:40.078882 11053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:40.080216 11053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:40.081378 11053 hybrid_clock.cc:648] HybridClock initialized: now 1790651140081365 us; error 30 us; skew 500 ppm
Sep 29 03:05:40 dist-test-slave-bdps krb5kdc[11040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651140, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:40.084424 11053 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:05:40.085548 11053 webserver.cc:460] Webserver started at http://127.7.209.126:35511/ using document root <none> and password file <none>
I20260929 03:05:40.086210 11053 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:40.086279 11053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:40.086513 11053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:40.088917 11053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "b086dbe34fb048de85ec235469d31eff"
format_stamp: "Formatted at 2026-09-29 03:05:40 on dist-test-slave-bdps"
server_key: "75849b5470c32c74bec96504f5da83f0"
server_key_iv: "51919a528731a5cf0ac31ff80207d0fa"
server_key_version: "encryptionkey@0"
I20260929 03:05:40.089521 11053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "b086dbe34fb048de85ec235469d31eff"
format_stamp: "Formatted at 2026-09-29 03:05:40 on dist-test-slave-bdps"
server_key: "75849b5470c32c74bec96504f5da83f0"
server_key_iv: "51919a528731a5cf0ac31ff80207d0fa"
server_key_version: "encryptionkey@0"
I20260929 03:05:40.093654 11053 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:40.096222 11068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:40.097499 11053 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 03:05:40.097637 11053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "b086dbe34fb048de85ec235469d31eff"
format_stamp: "Formatted at 2026-09-29 03:05:40 on dist-test-slave-bdps"
server_key: "75849b5470c32c74bec96504f5da83f0"
server_key_iv: "51919a528731a5cf0ac31ff80207d0fa"
server_key_version: "encryptionkey@0"
I20260929 03:05:40.097748 11053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:40.120434 11053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:40.124794 11053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:40.125032 11053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:40.132912 11053 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:35239
I20260929 03:05:40.132925 11120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:35239 every 8 connection(s)
I20260929 03:05:40.134018 11053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:40.135608 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 11053
I20260929 03:05:40.135730 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:40.136046 8005 external_mini_cluster.cc:1428] Setting key 5faeb17e5ae9065e94e34f2edff0a9da
I20260929 03:05:40.137534 11121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:40.145566 11121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff: Bootstrap starting.
Sep 29 03:05:40 dist-test-slave-bdps krb5kdc[11040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:40.148828 11121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:40.149909 11121 log.cc:824] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:40.152597 11121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff: No bootstrap required, opened a new log
I20260929 03:05:40.154243 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:40.137689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45154 (local address 127.7.209.126:35239)
0929 03:05:40.138122 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:40.138142 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:40.138190 (+ 48us) server_negotiation.cc:407] Connection header received
0929 03:05:40.139130 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:40.139155 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:40.139622 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:40.140034 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:40.140741 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.142368 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.143553 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.144305 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.144468 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:40.147698 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:40.147705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:40.147724 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:40.147760 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:40.150571 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:40.151131 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:40.151137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:40.151145 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:40.151288 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:40.151586 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:40.151590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:40.151592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:40.152224 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:40.152430 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:40.152749 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 03:05:40.153125 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":122,"threads_started":1}
I20260929 03:05:40.156857 11121 raft_consensus.cc:348] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b086dbe34fb048de85ec235469d31eff" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 35239 } }
I20260929 03:05:40.157153 11121 raft_consensus.cc:374] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:40.157219 11121 raft_consensus.cc:729] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b086dbe34fb048de85ec235469d31eff, State: Initialized, Role: FOLLOWER
I20260929 03:05:40.157760 11121 consensus_queue.cc:260] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b086dbe34fb048de85ec235469d31eff" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 35239 } }
I20260929 03:05:40.157917 11121 raft_consensus.cc:388] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:40.157985 11121 raft_consensus.cc:482] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:40.158097 11121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:40.159209 11121 raft_consensus.cc:504] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b086dbe34fb048de85ec235469d31eff" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 35239 } }
I20260929 03:05:40.159586 11121 leader_election.cc:302] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b086dbe34fb048de85ec235469d31eff; no voters:
I20260929 03:05:40.159926 11121 leader_election.cc:288] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:40.160044 11126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:40.160281 11126 raft_consensus.cc:686] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [term 1 LEADER]: Becoming Leader. State: Replica: b086dbe34fb048de85ec235469d31eff, State: Running, Role: LEADER
I20260929 03:05:40.160629 11126 consensus_queue.cc:237] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b086dbe34fb048de85ec235469d31eff" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 35239 } }
I20260929 03:05:40.161419 11121 sys_catalog.cc:564] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:40.162528 11127 sys_catalog.cc:455] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b086dbe34fb048de85ec235469d31eff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b086dbe34fb048de85ec235469d31eff" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 35239 } } }
I20260929 03:05:40.162679 11127 sys_catalog.cc:458] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:40.163084 11128 sys_catalog.cc:455] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [sys.catalog]: SysCatalogTable state changed. Reason: New leader b086dbe34fb048de85ec235469d31eff. Latest consensus state: current_term: 1 leader_uuid: "b086dbe34fb048de85ec235469d31eff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b086dbe34fb048de85ec235469d31eff" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 35239 } } }
I20260929 03:05:40.163249 11128 sys_catalog.cc:458] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:40.163329 11135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:40.166754 11135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:40.173149 11135 catalog_manager.cc:1347] Generated new cluster ID: b4923a3669da40e0a501d4ce8dc5f1d0
I20260929 03:05:40.173233 11135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:40.190659 11135 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:40.192173 11135 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:40.198269 11135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b086dbe34fb048de85ec235469d31eff: Generated new TSK 0
I20260929 03:05:40.199460 11135 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:05:40.234411 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:35239
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:40.360776 11147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:40.361111 11147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:40.361167 11147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:40.365559 11147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:40.365656 11147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:40.365732 11147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:40.371411 11147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:35239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:40.372674 11147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:40.373890 11147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:40.384337 11155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:40.384346 11152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:40.384347 11153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:40.384516 11147 server_base.cc:1034] running on GCE node
I20260929 03:05:40.385077 11147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:40.385732 11147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:40.386880 11147 hybrid_clock.cc:648] HybridClock initialized: now 1790651140386865 us; error 29 us; skew 500 ppm
Sep 29 03:05:40 dist-test-slave-bdps krb5kdc[11040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651140, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:40.389739 11147 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:05:40.390637 11147 webserver.cc:460] Webserver started at http://127.7.209.65:40541/ using document root <none> and password file <none>
I20260929 03:05:40.391258 11147 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:40.391321 11147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:40.391536 11147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:40.393839 11147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "e6a380d3bab24bc0a1486df55c3f96f2"
format_stamp: "Formatted at 2026-09-29 03:05:40 on dist-test-slave-bdps"
server_key: "53754cd43dbe8504f2df8713d0c7b865"
server_key_iv: "77b46b1a88226625da6fc1549df9cfdb"
server_key_version: "encryptionkey@0"
I20260929 03:05:40.394450 11147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "e6a380d3bab24bc0a1486df55c3f96f2"
format_stamp: "Formatted at 2026-09-29 03:05:40 on dist-test-slave-bdps"
server_key: "53754cd43dbe8504f2df8713d0c7b865"
server_key_iv: "77b46b1a88226625da6fc1549df9cfdb"
server_key_version: "encryptionkey@0"
I20260929 03:05:40.398763 11147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:40.401552 11162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:40.403002 11147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:40.403118 11147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "e6a380d3bab24bc0a1486df55c3f96f2"
format_stamp: "Formatted at 2026-09-29 03:05:40 on dist-test-slave-bdps"
server_key: "53754cd43dbe8504f2df8713d0c7b865"
server_key_iv: "77b46b1a88226625da6fc1549df9cfdb"
server_key_version: "encryptionkey@0"
I20260929 03:05:40.403231 11147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:40.445914 11147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:40.449929 11147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:40.450152 11147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:40.450927 11147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:40.451959 11147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:40.452013 11147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:40.452061 11147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:40.452090 11147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:40.461820 11147 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:33873
I20260929 03:05:40.461844 11275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:33873 every 8 connection(s)
I20260929 03:05:40.462970 11147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:40.472429 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 11147
I20260929 03:05:40.472594 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:05:40.472990 8005 external_mini_cluster.cc:1428] Setting key 795f66fe1794af2ed8f5ad39faed924f
Sep 29 03:05:40 dist-test-slave-bdps krb5kdc[11040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651140, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:40.478790 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:40.465189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:56641 (local address 127.7.209.126:35239)
0929 03:05:40.465343 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:40.465346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:40.466419 (+ 1073us) server_negotiation.cc:407] Connection header received
0929 03:05:40.467885 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:40.467888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:40.467947 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:40.468030 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:40.469436 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.470110 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.471299 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.471553 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.471632 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:40.474619 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:40.474622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:40.474624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:40.474656 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:40.476242 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:40.476838 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:40.476843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:40.476845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:40.477023 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:40.477397 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:40.477401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:40.477402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:40.477680 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:40.477783 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:40.478400 (+ 617us) server_negotiation.cc:299] Negotiation successful
0929 03:05:40.478575 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 03:05:40.479996 11278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:40.465707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:35239 (local address 127.7.209.65:56641)
0929 03:05:40.466239 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:40.466290 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:40.467658 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:40.468246 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:40.468254 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:40.468688 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:40.469301 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:40.469318 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.470213 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.470216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:40.471195 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:40.471204 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.471682 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.471684 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:40.471837 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:40.472485 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:40.472506 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:40.474465 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:40.476383 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:40.476390 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:40.476409 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:40.476743 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:40.477112 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:40.477114 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:40.477116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:40.477295 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:40.477787 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:40.477796 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:40.478131 (+ 335us) client_negotiation.cc:769] Sending connection context
0929 03:05:40.478342 (+ 211us) client_negotiation.cc:240] Negotiation successful
0929 03:05:40.478818 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":133,"threads_started":1}
I20260929 03:05:40.481588 11276 heartbeater.cc:347] Connected to a master server at 127.7.209.126:35239
I20260929 03:05:40.481992 11276 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:40.482852 11276 heartbeater.cc:511] Master 127.7.209.126:35239 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:05:40.485000 11085 ts_manager.cc:194] Registered new tserver with Master: e6a380d3bab24bc0a1486df55c3f96f2 (127.7.209.65:33873)
I20260929 03:05:40.486577 11085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:56641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:05:40.504001 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:35239
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:40.629459 11281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:40.629808 11281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:40.629858 11281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:40.633967 11281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:40.634042 11281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:40.634099 11281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:40.639660 11281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:35239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:40.641202 11281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:40.642524 11281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:40.653391 11286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:40.653379 11289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:40.653378 11287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:40.653798 11281 server_base.cc:1034] running on GCE node
I20260929 03:05:40.654254 11281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:40.654932 11281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:40.656108 11281 hybrid_clock.cc:648] HybridClock initialized: now 1790651140656091 us; error 30 us; skew 500 ppm
Sep 29 03:05:40 dist-test-slave-bdps krb5kdc[11040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651140, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:40.659013 11281 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:05:40.659938 11281 webserver.cc:460] Webserver started at http://127.7.209.66:35935/ using document root <none> and password file <none>
I20260929 03:05:40.660559 11281 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:40.660647 11281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:40.660893 11281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:40.663445 11281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "5cf5982daf9d480ea8171fef30847e80"
format_stamp: "Formatted at 2026-09-29 03:05:40 on dist-test-slave-bdps"
server_key: "83a30f74b3bf52fe5227c00866f23f55"
server_key_iv: "c796784f24975280bed510947a223728"
server_key_version: "encryptionkey@0"
I20260929 03:05:40.664065 11281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "5cf5982daf9d480ea8171fef30847e80"
format_stamp: "Formatted at 2026-09-29 03:05:40 on dist-test-slave-bdps"
server_key: "83a30f74b3bf52fe5227c00866f23f55"
server_key_iv: "c796784f24975280bed510947a223728"
server_key_version: "encryptionkey@0"
I20260929 03:05:40.668540 11281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260929 03:05:40.671360 11296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:40.672870 11281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:05:40.672986 11281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "5cf5982daf9d480ea8171fef30847e80"
format_stamp: "Formatted at 2026-09-29 03:05:40 on dist-test-slave-bdps"
server_key: "83a30f74b3bf52fe5227c00866f23f55"
server_key_iv: "c796784f24975280bed510947a223728"
server_key_version: "encryptionkey@0"
I20260929 03:05:40.673099 11281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:40.695978 11281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:40.699261 11281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:40.699429 11281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:40.700032 11281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:40.700927 11281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:40.700975 11281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:40.701017 11281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:40.701037 11281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:40.711063 11281 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:33485
I20260929 03:05:40.711087 11409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:33485 every 8 connection(s)
I20260929 03:05:40.712113 11281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:40.721760 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 11281
I20260929 03:05:40.721899 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:05:40.722275 8005 external_mini_cluster.cc:1428] Setting key a989255e999578d4780dea224cd8157f
Sep 29 03:05:40 dist-test-slave-bdps krb5kdc[11040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651140, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:40.729396 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:40.714170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:49123 (local address 127.7.209.126:35239)
0929 03:05:40.714314 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:40.714318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:40.715440 (+ 1122us) server_negotiation.cc:407] Connection header received
0929 03:05:40.717259 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:40.717263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:40.717335 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:40.717413 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:40.718849 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.719435 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.720672 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.720909 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.721008 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:40.725726 (+ 4718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:40.725729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:40.725731 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:40.725758 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:40.727187 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:40.727754 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:40.727756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:40.727758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:40.727832 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:40.728162 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:40.728164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:40.728166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:40.728334 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:40.728403 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:40.729143 (+ 740us) server_negotiation.cc:299] Negotiation successful
0929 03:05:40.729259 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:05:40.730722 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:40.714673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:35239 (local address 127.7.209.66:49123)
0929 03:05:40.715256 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:40.715306 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:40.716968 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:40.717670 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:40.717682 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:40.718115 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:40.718687 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:40.718703 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.719560 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.719564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:40.720547 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:40.720557 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.721071 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.721076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:40.721268 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:40.722032 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:40.722052 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:40.723852 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:40.727302 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:40.727308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:40.727324 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:40.727670 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:40.727902 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:40.727905 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:40.727907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:40.728082 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:40.728417 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:40.728423 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:40.728765 (+ 342us) client_negotiation.cc:769] Sending connection context
0929 03:05:40.728991 (+ 226us) client_negotiation.cc:240] Negotiation successful
0929 03:05:40.729378 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":144,"threads_started":1}
I20260929 03:05:40.732015 11410 heartbeater.cc:347] Connected to a master server at 127.7.209.126:35239
I20260929 03:05:40.732324 11410 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:40.733026 11410 heartbeater.cc:511] Master 127.7.209.126:35239 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:05:40.734340 11085 ts_manager.cc:194] Registered new tserver with Master: 5cf5982daf9d480ea8171fef30847e80 (127.7.209.66:33485)
I20260929 03:05:40.734953 11085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:49123
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:05:40.751943 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:35239
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:40.880023 11415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:40.880357 11415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:40.880406 11415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:40.884687 11415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:40.884759 11415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:40.884840 11415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:40.890664 11415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:35239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:40.891963 11415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:40.893229 11415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:40.903903 11423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:40.903934 11420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:40.903920 11421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:40.904076 11415 server_base.cc:1034] running on GCE node
I20260929 03:05:40.904582 11415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:40.905292 11415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:40.906447 11415 hybrid_clock.cc:648] HybridClock initialized: now 1790651140906429 us; error 32 us; skew 500 ppm
Sep 29 03:05:40 dist-test-slave-bdps krb5kdc[11040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651140, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:40.909410 11415 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:05:40.910406 11415 webserver.cc:460] Webserver started at http://127.7.209.67:38257/ using document root <none> and password file <none>
I20260929 03:05:40.911022 11415 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:40.911089 11415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:40.911304 11415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:40.913748 11415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "9c40414b40c447f0b7f465fa8f1fc2b6"
format_stamp: "Formatted at 2026-09-29 03:05:40 on dist-test-slave-bdps"
server_key: "b06a3bbdfa8093e24a3706729270899f"
server_key_iv: "fd2924fe3021cc117031dd19027f2423"
server_key_version: "encryptionkey@0"
I20260929 03:05:40.914407 11415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "9c40414b40c447f0b7f465fa8f1fc2b6"
format_stamp: "Formatted at 2026-09-29 03:05:40 on dist-test-slave-bdps"
server_key: "b06a3bbdfa8093e24a3706729270899f"
server_key_iv: "fd2924fe3021cc117031dd19027f2423"
server_key_version: "encryptionkey@0"
I20260929 03:05:40.919111 11415 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260929 03:05:40.921916 11430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:40.923350 11415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:05:40.923465 11415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "9c40414b40c447f0b7f465fa8f1fc2b6"
format_stamp: "Formatted at 2026-09-29 03:05:40 on dist-test-slave-bdps"
server_key: "b06a3bbdfa8093e24a3706729270899f"
server_key_iv: "fd2924fe3021cc117031dd19027f2423"
server_key_version: "encryptionkey@0"
I20260929 03:05:40.923571 11415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:40.944973 11415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:40.947788 11415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:40.947959 11415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:40.948575 11415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:40.949442 11415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:40.949491 11415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:40.949544 11415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:40.949573 11415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:40.959631 11415 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:38647
I20260929 03:05:40.959658 11543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:38647 every 8 connection(s)
I20260929 03:05:40.960812 11415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:40.969846 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 11415
I20260929 03:05:40.969983 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:05:40.970309 8005 external_mini_cluster.cc:1428] Setting key 9a401197d0aab9c8601d2c58b85aa3b5
Sep 29 03:05:40 dist-test-slave-bdps krb5kdc[11040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651140, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:40.975834 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:40.963032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:37297 (local address 127.7.209.126:35239)
0929 03:05:40.963193 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:40.963198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:40.964540 (+ 1342us) server_negotiation.cc:407] Connection header received
0929 03:05:40.966099 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:40.966104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:40.966171 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:40.966257 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:40.967615 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.968199 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.969408 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.969737 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.969857 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:40.972711 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:40.972714 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:40.972716 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:40.972742 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:40.974147 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:40.974579 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:40.974582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:40.974584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:40.974667 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:40.974922 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:40.974924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:40.974925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:40.975093 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:40.975173 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:40.975600 (+ 427us) server_negotiation.cc:299] Negotiation successful
0929 03:05:40.975702 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:05:40.976744 11546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:40.963699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:35239 (local address 127.7.209.67:37297)
0929 03:05:40.964355 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:40.964417 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:40.965866 (+ 1449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:40.966468 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:40.966476 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:40.966883 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:40.967449 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:40.967463 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.968322 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.968326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:40.969286 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:40.969295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:40.969888 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:40.969892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:40.970099 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:40.970751 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:40.970770 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:40.972566 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:40.974243 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:40.974247 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:40.974260 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:40.974502 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:40.974730 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:40.974732 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:40.974733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:40.974858 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:40.975165 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:40.975170 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:40.975412 (+ 242us) client_negotiation.cc:769] Sending connection context
0929 03:05:40.975570 (+ 158us) client_negotiation.cc:240] Negotiation successful
0929 03:05:40.975840 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":181,"threads_started":1}
I20260929 03:05:40.977880 11544 heartbeater.cc:347] Connected to a master server at 127.7.209.126:35239
I20260929 03:05:40.978160 11544 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:40.978668 11544 heartbeater.cc:511] Master 127.7.209.126:35239 requested a full tablet report, sending...
I20260929 03:05:40.979697 11085 ts_manager.cc:194] Registered new tserver with Master: 9c40414b40c447f0b7f465fa8f1fc2b6 (127.7.209.67:38647)
I20260929 03:05:40.980285 11085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:37297
I20260929 03:05:40.983891 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 03:05:40 dist-test-slave-bdps krb5kdc[11040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651140, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 03:05:41 dist-test-slave-bdps krb5kdc[11040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651140, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:41.010107 11124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:41.001929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45156 (local address 127.7.209.126:35239)
0929 03:05:41.002094 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:41.002099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:41.002249 (+ 150us) server_negotiation.cc:407] Connection header received
0929 03:05:41.002385 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:41.002388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:41.002441 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:41.002518 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:41.003232 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:41.003794 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:41.004586 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:41.004805 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:41.004873 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:41.007216 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:41.007219 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:41.007237 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:41.007263 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:41.008697 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:41.009030 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:41.009032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:41.009034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:41.009101 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:41.009279 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:41.009281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:41.009282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:41.009512 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:41.009656 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:41.009824 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 03:05:41.009924 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260929 03:05:41.010531 11085 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45156
I20260929 03:05:41.011345 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 11147
I20260929 03:05:41.017473 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 11281
I20260929 03:05:41.022301 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 11415
I20260929 03:05:41.026729 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 11053
2026-09-29T03:05:41Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2177 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:05:41 dist-test-slave-bdps krb5kdc[11556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:05:41 dist-test-slave-bdps krb5kdc[11556](info): set up 2 sockets
Sep 29 03:05:41 dist-test-slave-bdps krb5kdc[11556](info): commencing operation
krb5kdc: starting...
W20260929 03:05:42.038002 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:05:42 dist-test-slave-bdps krb5kdc[11556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:05:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:05:42.165122 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:35389
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:35389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:42.291775 11569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:42.292119 11569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:42.292181 11569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:42.296322 11569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:42.296386 11569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:42.296412 11569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:42.296433 11569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:42.296461 11569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:42.302096 11569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:35389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:35389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:42.303445 11569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:42.304769 11569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:42.314473 11575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:42.314484 11574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:42.314484 11577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:42.314545 11569 server_base.cc:1034] running on GCE node
I20260929 03:05:42.315420 11569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:42.316865 11569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:42.318030 11569 hybrid_clock.cc:648] HybridClock initialized: now 1790651142318016 us; error 41 us; skew 500 ppm
Sep 29 03:05:42 dist-test-slave-bdps krb5kdc[11556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:42.321246 11569 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:05:42.322448 11569 webserver.cc:460] Webserver started at http://127.7.209.126:34717/ using document root <none> and password file <none>
I20260929 03:05:42.323128 11569 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:42.323204 11569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:42.323452 11569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:42.325995 11569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "a322dda61eec4cecb69b1669c0f2aae4"
format_stamp: "Formatted at 2026-09-29 03:05:42 on dist-test-slave-bdps"
server_key: "a4092abcc881013fdd062c947898629b"
server_key_iv: "18c7f7fde60622497d6e8366b01b0d32"
server_key_version: "encryptionkey@0"
I20260929 03:05:42.326617 11569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "a322dda61eec4cecb69b1669c0f2aae4"
format_stamp: "Formatted at 2026-09-29 03:05:42 on dist-test-slave-bdps"
server_key: "a4092abcc881013fdd062c947898629b"
server_key_iv: "18c7f7fde60622497d6e8366b01b0d32"
server_key_version: "encryptionkey@0"
I20260929 03:05:42.330840 11569 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:42.333465 11584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:42.334854 11569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:42.334971 11569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "a322dda61eec4cecb69b1669c0f2aae4"
format_stamp: "Formatted at 2026-09-29 03:05:42 on dist-test-slave-bdps"
server_key: "a4092abcc881013fdd062c947898629b"
server_key_iv: "18c7f7fde60622497d6e8366b01b0d32"
server_key_version: "encryptionkey@0"
I20260929 03:05:42.335063 11569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:42.375345 11569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:42.379253 11569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:42.379462 11569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:42.387713 11569 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:35389
I20260929 03:05:42.387727 11636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:35389 every 8 connection(s)
I20260929 03:05:42.388866 11569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:42.392376 11637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:42.392741 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 11569
I20260929 03:05:42.392877 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:42.393252 8005 external_mini_cluster.cc:1428] Setting key 8e230096e2ab2b15f72c06be52b248b1
I20260929 03:05:42.400985 11637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4: Bootstrap starting.
Sep 29 03:05:42 dist-test-slave-bdps krb5kdc[11556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:42.404318 11637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:42.405192 11637 log.cc:824] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:42.407711 11637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4: No bootstrap required, opened a new log
I20260929 03:05:42.409523 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:42.395009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34868 (local address 127.7.209.126:35389)
0929 03:05:42.395503 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:42.395522 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:42.395567 (+ 45us) server_negotiation.cc:407] Connection header received
0929 03:05:42.396371 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:42.396393 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:42.396827 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:42.397194 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:42.398305 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:42.399453 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:42.400664 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:42.401223 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:42.401349 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:42.404350 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:42.404356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:42.404372 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:42.404401 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:42.406259 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:42.406690 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:42.406696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:42.406703 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:42.406851 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:42.407077 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:42.407080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:42.407081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:42.407630 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:42.407864 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:42.408109 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 03:05:42.408447 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":150,"threads_started":1}
I20260929 03:05:42.412345 11637 raft_consensus.cc:348] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a322dda61eec4cecb69b1669c0f2aae4" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 35389 } }
I20260929 03:05:42.412581 11637 raft_consensus.cc:374] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:42.412621 11637 raft_consensus.cc:729] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a322dda61eec4cecb69b1669c0f2aae4, State: Initialized, Role: FOLLOWER
I20260929 03:05:42.413024 11637 consensus_queue.cc:260] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a322dda61eec4cecb69b1669c0f2aae4" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 35389 } }
I20260929 03:05:42.413136 11637 raft_consensus.cc:388] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:42.413187 11637 raft_consensus.cc:482] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:42.413261 11637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:42.414433 11637 raft_consensus.cc:504] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a322dda61eec4cecb69b1669c0f2aae4" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 35389 } }
I20260929 03:05:42.414762 11637 leader_election.cc:302] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a322dda61eec4cecb69b1669c0f2aae4; no voters:
I20260929 03:05:42.415031 11637 leader_election.cc:288] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:42.415148 11642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:42.415369 11642 raft_consensus.cc:686] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [term 1 LEADER]: Becoming Leader. State: Replica: a322dda61eec4cecb69b1669c0f2aae4, State: Running, Role: LEADER
I20260929 03:05:42.415678 11642 consensus_queue.cc:237] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a322dda61eec4cecb69b1669c0f2aae4" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 35389 } }
I20260929 03:05:42.416325 11637 sys_catalog.cc:564] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:42.417414 11644 sys_catalog.cc:455] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a322dda61eec4cecb69b1669c0f2aae4. Latest consensus state: current_term: 1 leader_uuid: "a322dda61eec4cecb69b1669c0f2aae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a322dda61eec4cecb69b1669c0f2aae4" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 35389 } } }
I20260929 03:05:42.417568 11644 sys_catalog.cc:458] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:42.417431 11643 sys_catalog.cc:455] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a322dda61eec4cecb69b1669c0f2aae4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a322dda61eec4cecb69b1669c0f2aae4" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 35389 } } }
I20260929 03:05:42.417701 11643 sys_catalog.cc:458] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:42.418293 11651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:42.421913 11651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:42.428319 11651 catalog_manager.cc:1347] Generated new cluster ID: 17cd0e12f48b418f924936ac93827502
I20260929 03:05:42.428380 11651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:42.447090 11651 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:42.448258 11651 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:42.465981 11651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a322dda61eec4cecb69b1669c0f2aae4: Generated new TSK 0
I20260929 03:05:42.466898 11651 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:05:42.500718 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:35389
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:42.628787 11663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:42.629092 11663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:42.629134 11663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:42.633242 11663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:42.633308 11663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:42.633373 11663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:42.639015 11663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:35389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:42.640393 11663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:42.641683 11663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:05:42.652257 11663 server_base.cc:1034] running on GCE node
W20260929 03:05:42.652179 11668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:42.652199 11669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:42.652171 11671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:42.652861 11663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:42.653538 11663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:42.654711 11663 hybrid_clock.cc:648] HybridClock initialized: now 1790651142654691 us; error 32 us; skew 500 ppm
Sep 29 03:05:42 dist-test-slave-bdps krb5kdc[11556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:42.657660 11663 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:05:42.658569 11663 webserver.cc:460] Webserver started at http://127.7.209.65:37393/ using document root <none> and password file <none>
I20260929 03:05:42.659154 11663 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:42.659212 11663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:42.659400 11663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:42.661599 11663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "5539661bb9b84f04a92896d6750180d5"
format_stamp: "Formatted at 2026-09-29 03:05:42 on dist-test-slave-bdps"
server_key: "e999fae57db1352bcba047cd7016d4d3"
server_key_iv: "9d45ab3c67d1afcfa4081420b6c24fb2"
server_key_version: "encryptionkey@0"
I20260929 03:05:42.662238 11663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "5539661bb9b84f04a92896d6750180d5"
format_stamp: "Formatted at 2026-09-29 03:05:42 on dist-test-slave-bdps"
server_key: "e999fae57db1352bcba047cd7016d4d3"
server_key_iv: "9d45ab3c67d1afcfa4081420b6c24fb2"
server_key_version: "encryptionkey@0"
I20260929 03:05:42.666402 11663 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260929 03:05:42.669039 11678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:42.670420 11663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 03:05:42.670521 11663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "5539661bb9b84f04a92896d6750180d5"
format_stamp: "Formatted at 2026-09-29 03:05:42 on dist-test-slave-bdps"
server_key: "e999fae57db1352bcba047cd7016d4d3"
server_key_iv: "9d45ab3c67d1afcfa4081420b6c24fb2"
server_key_version: "encryptionkey@0"
I20260929 03:05:42.670611 11663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:42.698190 11663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:42.701998 11663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:42.702188 11663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:42.702875 11663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:42.703825 11663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:42.703876 11663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:42.703918 11663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:42.703938 11663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:42.713768 11663 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:37527
I20260929 03:05:42.713809 11791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:37527 every 8 connection(s)
I20260929 03:05:42.714850 11663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:42.718035 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 11663
I20260929 03:05:42.718236 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:05:42.718691 8005 external_mini_cluster.cc:1428] Setting key c3b3d0cf579b1f01e18a6de75a3cfef9
Sep 29 03:05:42 dist-test-slave-bdps krb5kdc[11556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:05:42.731478 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:42.717183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:42861 (local address 127.7.209.126:35389)
0929 03:05:42.717335 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:42.717340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:42.718679 (+ 1339us) server_negotiation.cc:407] Connection header received
0929 03:05:42.720502 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:42.720505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:42.720553 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:42.720616 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:42.722445 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:42.723043 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:42.724715 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:42.724960 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:42.725033 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:42.728029 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:42.728032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:42.728035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:42.728058 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:42.729419 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:42.729945 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:42.729948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:42.729950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:42.730024 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:42.730356 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:42.730359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:42.730360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:42.730539 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:42.730627 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:42.731153 (+ 526us) server_negotiation.cc:299] Negotiation successful
0929 03:05:42.731292 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 03:05:42.732383 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:42.717775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:35389 (local address 127.7.209.65:42861)
0929 03:05:42.718433 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:42.718510 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:42.720260 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:42.720868 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:42.720881 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:42.721454 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:42.722293 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:42.722317 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:42.723175 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:42.723180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:42.724590 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:42.724602 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:42.725116 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:42.725120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:42.725331 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:42.726041 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:42.726072 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:42.727864 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:42.729522 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:42.729529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:42.729546 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:42.729844 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:42.730103 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:42.730106 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:42.730108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:42.730262 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:42.730633 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:42.730648 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:42.730904 (+ 256us) client_negotiation.cc:769] Sending connection context
0929 03:05:42.731075 (+ 171us) client_negotiation.cc:240] Negotiation successful
0929 03:05:42.731423 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":148,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
I20260929 03:05:42.733759 11792 heartbeater.cc:347] Connected to a master server at 127.7.209.126:35389
I20260929 03:05:42.734071 11792 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:42.734670 11792 heartbeater.cc:511] Master 127.7.209.126:35389 requested a full tablet report, sending...
I20260929 03:05:42.736591 11601 ts_manager.cc:194] Registered new tserver with Master: 5539661bb9b84f04a92896d6750180d5 (127.7.209.65:37527)
I20260929 03:05:42.738483 11601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:42861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:05:42.747784 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:35389
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:42.870288 11797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:42.870687 11797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:42.870738 11797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:42.875103 11797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:42.875181 11797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:42.875248 11797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:42.881131 11797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:35389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:42.882625 11797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:42.883921 11797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:42.894686 11805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:42.894706 11802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:42.894685 11803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:42.894932 11797 server_base.cc:1034] running on GCE node
I20260929 03:05:42.895409 11797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:42.896080 11797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:42.897228 11797 hybrid_clock.cc:648] HybridClock initialized: now 1790651142897199 us; error 43 us; skew 500 ppm
Sep 29 03:05:42 dist-test-slave-bdps krb5kdc[11556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:42.900143 11797 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:05:42.901103 11797 webserver.cc:460] Webserver started at http://127.7.209.66:35629/ using document root <none> and password file <none>
I20260929 03:05:42.901763 11797 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:42.901839 11797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:42.902040 11797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:42.904424 11797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "38c09eaa12e8470aaba9ffa17d4e665a"
format_stamp: "Formatted at 2026-09-29 03:05:42 on dist-test-slave-bdps"
server_key: "6da53e00f87a40699d7958bd75a4aa8f"
server_key_iv: "2dad8233fee487d560c38a4726af9189"
server_key_version: "encryptionkey@0"
I20260929 03:05:42.905087 11797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "38c09eaa12e8470aaba9ffa17d4e665a"
format_stamp: "Formatted at 2026-09-29 03:05:42 on dist-test-slave-bdps"
server_key: "6da53e00f87a40699d7958bd75a4aa8f"
server_key_iv: "2dad8233fee487d560c38a4726af9189"
server_key_version: "encryptionkey@0"
I20260929 03:05:42.909479 11797 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 03:05:42.912293 11812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:42.913872 11797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:42.913990 11797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "38c09eaa12e8470aaba9ffa17d4e665a"
format_stamp: "Formatted at 2026-09-29 03:05:42 on dist-test-slave-bdps"
server_key: "6da53e00f87a40699d7958bd75a4aa8f"
server_key_iv: "2dad8233fee487d560c38a4726af9189"
server_key_version: "encryptionkey@0"
I20260929 03:05:42.914106 11797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:42.925915 11797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:42.928913 11797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:42.929097 11797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:42.929790 11797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:42.930689 11797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:42.930737 11797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:42.930783 11797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:42.930810 11797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:42.940389 11797 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:36499
I20260929 03:05:42.940414 11925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:36499 every 8 connection(s)
I20260929 03:05:42.941308 11797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:42.944700 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 11797
I20260929 03:05:42.944806 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:05:42.945094 8005 external_mini_cluster.cc:1428] Setting key 478f142ad2506a43b75372975f8e80a5
Sep 29 03:05:42 dist-test-slave-bdps krb5kdc[11556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651142, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:42.955792 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:42.943243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:59101 (local address 127.7.209.126:35389)
0929 03:05:42.943372 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:42.943376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:42.944265 (+ 889us) server_negotiation.cc:407] Connection header received
0929 03:05:42.945801 (+ 1536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:42.945803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:42.945863 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:42.945952 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:42.947212 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:42.947981 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:42.949313 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:42.949543 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:42.949690 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:42.952270 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:42.952273 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:42.952275 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:42.952300 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:42.953678 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:42.954229 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:42.954231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:42.954233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:42.954315 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:42.954671 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:42.954673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:42.954674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:42.954846 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:42.954921 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:42.955536 (+ 615us) server_negotiation.cc:299] Negotiation successful
0929 03:05:42.955648 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:05:42.957011 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:42.943668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:35389 (local address 127.7.209.66:59101)
0929 03:05:42.944111 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:42.944162 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:42.945558 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:42.946115 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:42.946123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:42.946560 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:42.947068 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:42.947082 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:42.948086 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:42.948089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:42.949198 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:42.949209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:42.949703 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:42.949707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:42.949883 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:42.950392 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:42.950411 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:42.952105 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:42.953783 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:42.953789 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:42.953808 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:42.954133 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:42.954392 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:42.954395 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:42.954397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:42.954571 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:42.954940 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:42.954947 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:42.955280 (+ 333us) client_negotiation.cc:769] Sending connection context
0929 03:05:42.955511 (+ 231us) client_negotiation.cc:240] Negotiation successful
0929 03:05:42.955901 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":107,"threads_started":1}
I20260929 03:05:42.958369 11926 heartbeater.cc:347] Connected to a master server at 127.7.209.126:35389
I20260929 03:05:42.958627 11926 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
I20260929 03:05:42.959204 11926 heartbeater.cc:511] Master 127.7.209.126:35389 requested a full tablet report, sending...
I20260929 03:05:42.960445 11601 ts_manager.cc:194] Registered new tserver with Master: 38c09eaa12e8470aaba9ffa17d4e665a (127.7.209.66:36499)
I20260929 03:05:42.961256 11601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:59101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:05:42.974392 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:35389
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:43503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:43.120453 11931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:43.120820 11931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:43.120885 11931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:43.125260 11931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:43.125362 11931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:43.125452 11931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:43.132942 11931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:43503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:35389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:43.134641 11931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:43.136214 11931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:43.151491 11936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:43.151491 11939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:43.151491 11937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:43.152357 11931 server_base.cc:1034] running on GCE node
I20260929 03:05:43.153059 11931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:43.154248 11931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:43.155478 11931 hybrid_clock.cc:648] HybridClock initialized: now 1790651143155432 us; error 63 us; skew 500 ppm
Sep 29 03:05:43 dist-test-slave-bdps krb5kdc[11556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:43.160372 11931 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:05:43.162285 11931 webserver.cc:460] Webserver started at http://127.7.209.67:33207/ using document root <none> and password file <none>
I20260929 03:05:43.163491 11931 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:43.163659 11931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:43.163933 11931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:43.167539 11931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "cd0b499637644acf98dd49e1e96689e8"
format_stamp: "Formatted at 2026-09-29 03:05:43 on dist-test-slave-bdps"
server_key: "d978d8a48ff4fc93cc874cfe8125fb42"
server_key_iv: "681717b16aeffb39d07c55092038b586"
server_key_version: "encryptionkey@0"
I20260929 03:05:43.169104 11931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "cd0b499637644acf98dd49e1e96689e8"
format_stamp: "Formatted at 2026-09-29 03:05:43 on dist-test-slave-bdps"
server_key: "d978d8a48ff4fc93cc874cfe8125fb42"
server_key_iv: "681717b16aeffb39d07c55092038b586"
server_key_version: "encryptionkey@0"
I20260929 03:05:43.183990 11931 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.004s
I20260929 03:05:43.188261 11946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:43.190461 11931 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 03:05:43.190666 11931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "cd0b499637644acf98dd49e1e96689e8"
format_stamp: "Formatted at 2026-09-29 03:05:43 on dist-test-slave-bdps"
server_key: "d978d8a48ff4fc93cc874cfe8125fb42"
server_key_iv: "681717b16aeffb39d07c55092038b586"
server_key_version: "encryptionkey@0"
I20260929 03:05:43.190826 11931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:43.212042 11931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:43.217221 11931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:43.217476 11931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:43.218451 11931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:43.219863 11931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:43.219920 11931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:43.219967 11931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:43.219987 11931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:43.234200 11931 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:46865
I20260929 03:05:43.234230 12059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:46865 every 8 connection(s)
I20260929 03:05:43.235693 11931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:43.243013 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 11931
I20260929 03:05:43.243245 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:05:43.243757 8005 external_mini_cluster.cc:1428] Setting key f352f28ea5ded6b9e6ad66d4ab0fd168
Sep 29 03:05:43 dist-test-slave-bdps krb5kdc[11556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:43.254208 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:43.238246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:40853 (local address 127.7.209.126:35389)
0929 03:05:43.238413 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:43.238418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:43.239639 (+ 1221us) server_negotiation.cc:407] Connection header received
0929 03:05:43.241861 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:43.241866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:43.241955 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:43.242084 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:43.244327 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:43.244996 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:43.246271 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:43.246497 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:43.246592 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:43.250193 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:43.250197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:43.250206 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:43.250242 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:43.252300 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:43.252764 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:43.252767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:43.252768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:43.252846 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:43.253130 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:43.253137 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:43.253139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:43.253316 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:43.253390 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:43.253934 (+ 544us) server_negotiation.cc:299] Negotiation successful
0929 03:05:43.254043 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 03:05:43.255285 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:43.238746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:35389 (local address 127.7.209.67:40853)
0929 03:05:43.239382 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:43.239477 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:43.241449 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:43.242445 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:43.242462 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:43.243144 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:43.244150 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:43.244168 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:43.245122 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:43.245125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:43.246159 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:43.246169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:43.246647 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:43.246651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:43.246838 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:43.247483 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:43.247513 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:43.249991 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:43.252389 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:43.252393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:43.252407 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:43.252668 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:43.252919 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:43.252921 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:43.252923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:43.253056 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:43.253386 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:43.253391 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:43.253690 (+ 299us) client_negotiation.cc:769] Sending connection context
0929 03:05:43.253876 (+ 186us) client_negotiation.cc:240] Negotiation successful
0929 03:05:43.254183 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":121,"threads_started":1}
I20260929 03:05:43.256791 12060 heartbeater.cc:347] Connected to a master server at 127.7.209.126:35389
I20260929 03:05:43.257051 12060 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:43.258152 12060 heartbeater.cc:511] Master 127.7.209.126:35389 requested a full tablet report, sending...
I20260929 03:05:43.259392 11601 ts_manager.cc:194] Registered new tserver with Master: cd0b499637644acf98dd49e1e96689e8 (127.7.209.67:46865)
I20260929 03:05:43.260279 11601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:40853
I20260929 03:05:43.270188 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 03:05:43 dist-test-slave-bdps krb5kdc[11556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260929 03:05:43.380414 8005 mini_kdc.cc:317] Time spent kinit for test-admin: real 0.109s user 0.001s sys 0.006s
Sep 29 03:05:43 dist-test-slave-bdps krb5kdc[11556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
I20260929 03:05:43.399178 12067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:43.383413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34420 (local address 127.7.209.65:37527)
0929 03:05:43.383846 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:43.383856 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:43.383891 (+ 35us) server_negotiation.cc:407] Connection header received
0929 03:05:43.384123 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:43.384131 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:43.384530 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:43.384805 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:43.385942 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:43.387070 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:43.388831 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:43.389247 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:43.389333 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:43.393473 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:43.393483 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:43.393498 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:43.393537 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:43.396583 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:43.397433 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:43.397440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:43.397441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:43.397544 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:43.397887 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:43.397891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:43.397892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:43.398301 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:43.398543 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:43.398791 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 03:05:43.398979 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":115,"threads_started":1}
Sep 29 03:05:43 dist-test-slave-bdps krb5kdc[11556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:43.415432 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:43.403588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.7.209.126:35389)
0929 03:05:43.403815 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:43.403820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:43.403957 (+ 137us) server_negotiation.cc:407] Connection header received
0929 03:05:43.404072 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:43.404075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:43.404128 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:43.404211 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:43.405000 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:43.406124 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:43.407296 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:43.407669 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:43.407803 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:43.411366 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:43.411369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:43.411371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:43.411406 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:43.413416 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:43.414125 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:43.414127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:43.414129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:43.414208 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:43.414415 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:43.414417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:43.414418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:43.414770 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:43.414963 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:43.415125 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 03:05:43.415302 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260929 03:05:43.416014 11601 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34876
I20260929 03:05:43.416882 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 11663
I20260929 03:05:43.424865 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 11797
I20260929 03:05:43.432235 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 11931
I20260929 03:05:43.439478 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 11569
2026-09-29T03:05:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2417 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:05:43 dist-test-slave-bdps krb5kdc[12072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:05:43 dist-test-slave-bdps krb5kdc[12072](info): set up 2 sockets
Sep 29 03:05:43 dist-test-slave-bdps krb5kdc[12072](info): commencing operation
krb5kdc: starting...
W20260929 03:05:44.526270 8005 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.002s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:05:44 dist-test-slave-bdps krb5kdc[12072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:05:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:05:44.658295 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:41665
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:41665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:44.799929 12085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:44.800328 12085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:44.800397 12085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:44.804713 12085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:44.804793 12085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:44.804822 12085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:44.804845 12085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:44.804873 12085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:44.810681 12085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:41665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:41665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:44.812031 12085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:44.813412 12085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:44.823360 12090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:44.823374 12093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:44.823367 12091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:44.823586 12085 server_base.cc:1034] running on GCE node
I20260929 03:05:44.824321 12085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:44.825626 12085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:44.826794 12085 hybrid_clock.cc:648] HybridClock initialized: now 1790651144826772 us; error 39 us; skew 500 ppm
Sep 29 03:05:44 dist-test-slave-bdps krb5kdc[12072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651144, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:44.834261 12085 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:05:44.835218 12085 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:44.835301 12085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:44.835553 12085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:44.838341 12085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "3f02e8bbc64f4748b458a929d7286e81"
format_stamp: "Formatted at 2026-09-29 03:05:44 on dist-test-slave-bdps"
server_key: "1643d31b9ef99ebc571e50e6c81e2629"
server_key_iv: "30a0a090c4ddcefcee6cc56f1daf3fc2"
server_key_version: "encryptionkey@0"
I20260929 03:05:44.839036 12085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "3f02e8bbc64f4748b458a929d7286e81"
format_stamp: "Formatted at 2026-09-29 03:05:44 on dist-test-slave-bdps"
server_key: "1643d31b9ef99ebc571e50e6c81e2629"
server_key_iv: "30a0a090c4ddcefcee6cc56f1daf3fc2"
server_key_version: "encryptionkey@0"
I20260929 03:05:44.843852 12085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260929 03:05:44.846588 12099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:44.848081 12085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:44.848214 12085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "3f02e8bbc64f4748b458a929d7286e81"
format_stamp: "Formatted at 2026-09-29 03:05:44 on dist-test-slave-bdps"
server_key: "1643d31b9ef99ebc571e50e6c81e2629"
server_key_iv: "30a0a090c4ddcefcee6cc56f1daf3fc2"
server_key_version: "encryptionkey@0"
I20260929 03:05:44.848315 12085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:44.863624 12085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:44.867045 12085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:44.867223 12085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:44.874926 12085 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:41665
I20260929 03:05:44.875223 12151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:41665 every 8 connection(s)
I20260929 03:05:44.875723 12085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:44.879017 12152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:44.885212 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 12085
I20260929 03:05:44.885352 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:44.885715 8005 external_mini_cluster.cc:1428] Setting key 3c69f931b4d3b4967d347acce2340c03
I20260929 03:05:44.886657 12152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81: Bootstrap starting.
I20260929 03:05:44.889976 12152 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:44.891004 12152 log.cc:824] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:44.893911 12152 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81: No bootstrap required, opened a new log
Sep 29 03:05:44 dist-test-slave-bdps krb5kdc[12072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:44.899993 12152 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f02e8bbc64f4748b458a929d7286e81" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41665 } }
I20260929 03:05:44.900256 12152 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:44.900326 12152 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f02e8bbc64f4748b458a929d7286e81, State: Initialized, Role: FOLLOWER
I20260929 03:05:44.901026 12152 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f02e8bbc64f4748b458a929d7286e81" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41665 } }
I20260929 03:05:44.901167 12152 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:44.901232 12152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:44.901376 12152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:44.902661 12152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f02e8bbc64f4748b458a929d7286e81" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41665 } }
I20260929 03:05:44.903069 12152 leader_election.cc:302] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f02e8bbc64f4748b458a929d7286e81; no voters:
I20260929 03:05:44.903596 12152 leader_election.cc:288] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:44.903797 12157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:44.903999 12157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [term 1 LEADER]: Becoming Leader. State: Replica: 3f02e8bbc64f4748b458a929d7286e81, State: Running, Role: LEADER
I20260929 03:05:44.904412 12157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f02e8bbc64f4748b458a929d7286e81" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41665 } }
I20260929 03:05:44.904975 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:44.887593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33568 (local address 127.7.209.126:41665)
0929 03:05:44.888131 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:44.888155 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:44.888216 (+ 61us) server_negotiation.cc:407] Connection header received
0929 03:05:44.889284 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:44.889314 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:44.889858 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:44.890327 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:44.891089 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:44.892684 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:44.893914 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:44.894725 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:44.894986 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:44.899166 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:44.899172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:44.899188 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:44.899216 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:44.901247 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:44.901806 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:44.901811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:44.901816 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:44.901991 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:44.902251 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:44.902253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:44.902254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:44.902716 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:44.902906 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:44.903170 (+ 264us) server_negotiation.cc:299] Negotiation successful
0929 03:05:44.903662 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":153,"threads_started":1}
I20260929 03:05:44.905360 12152 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:44.907050 12159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f02e8bbc64f4748b458a929d7286e81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f02e8bbc64f4748b458a929d7286e81" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41665 } } }
I20260929 03:05:44.907239 12159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:44.908555 12166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:44.908797 12157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f02e8bbc64f4748b458a929d7286e81. Latest consensus state: current_term: 1 leader_uuid: "3f02e8bbc64f4748b458a929d7286e81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f02e8bbc64f4748b458a929d7286e81" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41665 } } }
I20260929 03:05:44.909190 12157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:44.912065 12166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:44.917963 12166 catalog_manager.cc:1347] Generated new cluster ID: 8c464f23c77b47d49c107d7dc82a08e8
I20260929 03:05:44.918045 12166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:44.931414 12166 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:44.932430 12166 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:44.941021 12166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81: Generated new TSK 0
I20260929 03:05:44.941771 12166 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:05:44.974687 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:41665
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:45.104839 12178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:45.105127 12178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:45.105187 12178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:45.109331 12178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:45.109392 12178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:45.109468 12178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:45.114818 12178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:41665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:45.116168 12178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:45.117473 12178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:45.128522 12184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:45.128537 12186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:45.128556 12183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:45.128911 12178 server_base.cc:1034] running on GCE node
I20260929 03:05:45.129336 12178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:45.130051 12178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:45.131208 12178 hybrid_clock.cc:648] HybridClock initialized: now 1790651145131186 us; error 34 us; skew 500 ppm
Sep 29 03:05:45 dist-test-slave-bdps krb5kdc[12072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651145, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:45.134238 12178 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:05:45.135075 12178 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:45.135150 12178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:45.135391 12178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:45.137804 12178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "7532633e5c50435391a2943707ab1ca7"
format_stamp: "Formatted at 2026-09-29 03:05:45 on dist-test-slave-bdps"
server_key: "fffbc4adea13622257444522de1df202"
server_key_iv: "a172d69e419a48ab474c2c52a061f822"
server_key_version: "encryptionkey@0"
I20260929 03:05:45.138406 12178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "7532633e5c50435391a2943707ab1ca7"
format_stamp: "Formatted at 2026-09-29 03:05:45 on dist-test-slave-bdps"
server_key: "fffbc4adea13622257444522de1df202"
server_key_iv: "a172d69e419a48ab474c2c52a061f822"
server_key_version: "encryptionkey@0"
I20260929 03:05:45.142619 12178 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260929 03:05:45.145185 12192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:45.146529 12178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 03:05:45.146637 12178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "7532633e5c50435391a2943707ab1ca7"
format_stamp: "Formatted at 2026-09-29 03:05:45 on dist-test-slave-bdps"
server_key: "fffbc4adea13622257444522de1df202"
server_key_iv: "a172d69e419a48ab474c2c52a061f822"
server_key_version: "encryptionkey@0"
I20260929 03:05:45.146727 12178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:45.166975 12178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:45.170704 12178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:45.170913 12178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:45.171626 12178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:45.172600 12178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:45.172649 12178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:45.172708 12178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:45.172730 12178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:45.183601 12178 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:46847
I20260929 03:05:45.183630 12305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:46847 every 8 connection(s)
I20260929 03:05:45.184346 12178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:45.192360 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 12178
I20260929 03:05:45.192487 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:05:45.192878 8005 external_mini_cluster.cc:1428] Setting key d5d1ee87c03948087d6e6f08f437d828
Sep 29 03:05:45 dist-test-slave-bdps krb5kdc[12072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651145, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:45.199913 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.186440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:35479 (local address 127.7.209.126:41665)
0929 03:05:45.186591 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:45.186595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:45.187998 (+ 1403us) server_negotiation.cc:407] Connection header received
0929 03:05:45.189013 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:45.189017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:45.189084 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:45.189182 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:45.190576 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.191171 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.192401 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.192743 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.192857 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.196023 (+ 3166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:45.196026 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:45.196029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:45.196056 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:45.197687 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:45.198198 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:45.198202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:45.198204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:45.198313 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:45.198658 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:45.198661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:45.198663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:45.198914 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:45.199008 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:45.199551 (+ 543us) server_negotiation.cc:299] Negotiation successful
0929 03:05:45.199722 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 03:05:45.200940 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.186926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:41665 (local address 127.7.209.65:35479)
0929 03:05:45.187825 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:45.187873 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:45.188778 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:45.189379 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:45.189388 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:45.189848 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:45.190415 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.190430 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.191294 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.191297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.192276 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.192286 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.192909 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.192913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.193094 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.193785 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:45.193806 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:45.195844 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:45.197806 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:45.197812 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:45.197828 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:45.198101 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:45.198407 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:45.198409 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:45.198411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:45.198564 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:45.199003 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:45.199009 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:45.199303 (+ 294us) client_negotiation.cc:769] Sending connection context
0929 03:05:45.199482 (+ 179us) client_negotiation.cc:240] Negotiation successful
0929 03:05:45.199797 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":133,"threads_started":1}
I20260929 03:05:45.202440 12306 heartbeater.cc:347] Connected to a master server at 127.7.209.126:41665
I20260929 03:05:45.202775 12306 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:45.203419 12306 heartbeater.cc:511] Master 127.7.209.126:41665 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:05:45.205379 12116 ts_manager.cc:194] Registered new tserver with Master: 7532633e5c50435391a2943707ab1ca7 (127.7.209.65:46847)
I20260929 03:05:45.206831 12116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:35479
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:05:45.220356 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:41665
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:45.344157 12311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:45.344444 12311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:45.344502 12311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:45.348467 12311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:45.348520 12311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:45.348606 12311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:45.354001 12311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:41665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:45.355208 12311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:45.356465 12311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:45.367727 12317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:45.367812 12319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:45.367727 12316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:45.367969 12311 server_base.cc:1034] running on GCE node
I20260929 03:05:45.368530 12311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:45.369256 12311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:45.370409 12311 hybrid_clock.cc:648] HybridClock initialized: now 1790651145370376 us; error 45 us; skew 500 ppm
Sep 29 03:05:45 dist-test-slave-bdps krb5kdc[12072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651145, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:45.373523 12311 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:05:45.374392 12311 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:45.374459 12311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:45.374759 12311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:45.377195 12311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "72ae0485f9554c7b83b678cb412bf3a8"
format_stamp: "Formatted at 2026-09-29 03:05:45 on dist-test-slave-bdps"
server_key: "e36195e6cbce8bb32ebe3115c23b06d7"
server_key_iv: "6929aaefd5afa04b466bfb2fbfab9b55"
server_key_version: "encryptionkey@0"
I20260929 03:05:45.377840 12311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "72ae0485f9554c7b83b678cb412bf3a8"
format_stamp: "Formatted at 2026-09-29 03:05:45 on dist-test-slave-bdps"
server_key: "e36195e6cbce8bb32ebe3115c23b06d7"
server_key_iv: "6929aaefd5afa04b466bfb2fbfab9b55"
server_key_version: "encryptionkey@0"
I20260929 03:05:45.381947 12311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:05:45.384542 12325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:45.386040 12311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:45.386147 12311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "72ae0485f9554c7b83b678cb412bf3a8"
format_stamp: "Formatted at 2026-09-29 03:05:45 on dist-test-slave-bdps"
server_key: "e36195e6cbce8bb32ebe3115c23b06d7"
server_key_iv: "6929aaefd5afa04b466bfb2fbfab9b55"
server_key_version: "encryptionkey@0"
I20260929 03:05:45.386247 12311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:45.400556 12311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:45.404379 12311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:45.404589 12311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:45.405284 12311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:45.406354 12311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:45.406406 12311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:45.406457 12311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:45.406486 12311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:45.416076 12311 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:37893
I20260929 03:05:45.416093 12438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:37893 every 8 connection(s)
I20260929 03:05:45.416805 12311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:45.426658 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 12311
I20260929 03:05:45.426774 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:05:45.427078 8005 external_mini_cluster.cc:1428] Setting key c94bbfcce1e4a19904941b3fe8112cfd
Sep 29 03:05:45 dist-test-slave-bdps krb5kdc[12072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651145, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:45.431841 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.418889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:48921 (local address 127.7.209.126:41665)
0929 03:05:45.419028 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:45.419031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:45.420353 (+ 1322us) server_negotiation.cc:407] Connection header received
0929 03:05:45.421374 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:45.421378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:45.421446 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:45.421528 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:45.422927 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.423517 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.424739 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.424975 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.425067 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.427828 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:45.427831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:45.427834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:45.427862 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:45.429388 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:45.430026 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:45.430029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:45.430031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:45.430114 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:45.430494 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:45.430496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:45.430497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:45.430694 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:45.430772 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:45.431474 (+ 702us) server_negotiation.cc:299] Negotiation successful
0929 03:05:45.431650 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:05:45.432864 12441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.419330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:41665 (local address 127.7.209.66:48921)
0929 03:05:45.420184 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:45.420230 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:45.421137 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:45.421776 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:45.421785 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:45.422195 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:45.422771 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.422786 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.423640 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.423643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.424609 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.424618 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.425118 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.425123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.425303 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.425946 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:45.425963 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:45.427644 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:45.429507 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:45.429514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:45.429532 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:45.429930 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:45.430207 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:45.430210 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:45.430211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:45.430403 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:45.430785 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:45.430791 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:45.431157 (+ 366us) client_negotiation.cc:769] Sending connection context
0929 03:05:45.431394 (+ 237us) client_negotiation.cc:240] Negotiation successful
0929 03:05:45.431747 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":126,"threads_started":1}
I20260929 03:05:45.434293 12439 heartbeater.cc:347] Connected to a master server at 127.7.209.126:41665
I20260929 03:05:45.434574 12439 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:45.435113 12439 heartbeater.cc:511] Master 127.7.209.126:41665 requested a full tablet report, sending...
I20260929 03:05:45.436367 12116 ts_manager.cc:194] Registered new tserver with Master: 72ae0485f9554c7b83b678cb412bf3a8 (127.7.209.66:37893)
I20260929 03:05:45.437052 12116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:48921
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:05:45.457091 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:41665
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:45.582360 12444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:45.582672 12444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:45.582737 12444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:45.586805 12444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:45.586865 12444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:45.586944 12444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:45.592309 12444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:41665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:45.593539 12444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:45.594798 12444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:45.606195 12449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:45.606211 12452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:45.606251 12450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:45.606541 12444 server_base.cc:1034] running on GCE node
I20260929 03:05:45.607183 12444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:45.608012 12444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:45.609166 12444 hybrid_clock.cc:648] HybridClock initialized: now 1790651145609134 us; error 46 us; skew 500 ppm
Sep 29 03:05:45 dist-test-slave-bdps krb5kdc[12072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651145, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:45.612361 12444 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:05:45.613266 12444 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:45.613336 12444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:45.613575 12444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:45.616283 12444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "0d44d16a48a74f7883c46cbcfbaf2fea"
format_stamp: "Formatted at 2026-09-29 03:05:45 on dist-test-slave-bdps"
server_key: "377c62582460dcfdccdee1716fa5fec9"
server_key_iv: "322ff29228eb33f6023a7bdef04b79ae"
server_key_version: "encryptionkey@0"
I20260929 03:05:45.616882 12444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "0d44d16a48a74f7883c46cbcfbaf2fea"
format_stamp: "Formatted at 2026-09-29 03:05:45 on dist-test-slave-bdps"
server_key: "377c62582460dcfdccdee1716fa5fec9"
server_key_iv: "322ff29228eb33f6023a7bdef04b79ae"
server_key_version: "encryptionkey@0"
I20260929 03:05:45.621328 12444 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 03:05:45.624130 12458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:45.625568 12444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:05:45.625713 12444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "0d44d16a48a74f7883c46cbcfbaf2fea"
format_stamp: "Formatted at 2026-09-29 03:05:45 on dist-test-slave-bdps"
server_key: "377c62582460dcfdccdee1716fa5fec9"
server_key_iv: "322ff29228eb33f6023a7bdef04b79ae"
server_key_version: "encryptionkey@0"
I20260929 03:05:45.625818 12444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:45.645592 12444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:45.648967 12444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:45.649165 12444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:45.649919 12444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:45.650909 12444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:45.650960 12444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:45.651016 12444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:45.651044 12444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:45.660542 12444 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:46103
I20260929 03:05:45.660565 12571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:46103 every 8 connection(s)
I20260929 03:05:45.661229 12444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:45.664774 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 12444
I20260929 03:05:45.664906 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:05:45.665241 8005 external_mini_cluster.cc:1428] Setting key 1d5648720e4af6d7e6f4cb5b458fd4e3
Sep 29 03:05:45 dist-test-slave-bdps krb5kdc[12072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651145, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:45.675312 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.663251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:54565 (local address 127.7.209.126:41665)
0929 03:05:45.663399 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:45.663402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:45.664724 (+ 1322us) server_negotiation.cc:407] Connection header received
0929 03:05:45.665833 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:45.665836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:45.665907 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:45.665986 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:45.667349 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.668001 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.669211 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.669446 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.669537 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.672024 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:45.672028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:45.672030 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:45.672055 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:45.673291 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:45.673807 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:45.673811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:45.673813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:45.673892 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:45.674206 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:45.674210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:45.674212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:45.674397 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:45.674480 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:45.675004 (+ 524us) server_negotiation.cc:299] Negotiation successful
0929 03:05:45.675130 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 03:05:45.676252 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.663694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:41665 (local address 127.7.209.67:54565)
0929 03:05:45.664562 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:45.664608 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:45.665561 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:45.666208 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:45.666216 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:45.666621 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:45.667203 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.667219 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.668121 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.668124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.669090 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.669098 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.669585 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.669588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.669803 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.670364 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:45.670381 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:45.671866 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:45.673395 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:45.673400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:45.673416 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:45.673716 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:45.673971 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:45.673974 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:45.673976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:45.674112 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:45.674491 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:45.674498 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:45.674761 (+ 263us) client_negotiation.cc:769] Sending connection context
0929 03:05:45.674931 (+ 170us) client_negotiation.cc:240] Negotiation successful
0929 03:05:45.675293 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":126,"threads_started":1}
I20260929 03:05:45.677421 12572 heartbeater.cc:347] Connected to a master server at 127.7.209.126:41665
I20260929 03:05:45.677711 12572 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:45.678282 12572 heartbeater.cc:511] Master 127.7.209.126:41665 requested a full tablet report, sending...
I20260929 03:05:45.679548 12116 ts_manager.cc:194] Registered new tserver with Master: 0d44d16a48a74f7883c46cbcfbaf2fea (127.7.209.67:46103)
I20260929 03:05:45.680223 12116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:54565
I20260929 03:05:45.691282 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:05:45.700706 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.693729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33582 (local address 127.7.209.126:41665)
0929 03:05:45.693886 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:45.693891 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:45.694065 (+ 174us) server_negotiation.cc:407] Connection header received
0929 03:05:45.694178 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:45.694180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:45.694225 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:45.694291 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:45.695019 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.695560 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.696369 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.696587 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.696663 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.698013 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:45.698016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:45.698018 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:45.698043 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:45.699315 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:45.699691 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:45.699694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:45.699695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:45.699769 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:45.699964 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:45.699966 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:45.699967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:45.700199 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:45.700301 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:45.700449 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 03:05:45.700545 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:05:45.703537 12116 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:05:45.707243 12116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:45.723078 12584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.716992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:46847 (local address 127.0.0.1:43336)
0929 03:05:45.718168 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:45.718231 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:45.718385 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:45.719197 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:45.719200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:45.719215 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:45.719329 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.719335 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.720323 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.720327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.721541 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.721550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.722472 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.722475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.722585 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.722627 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 03:05:45.722751 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 03:05:45.722889 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"mutex_wait_us":1,"spinlock_wait_cycles":8448,"thread_start_us":149,"threads_started":1}
I20260929 03:05:45.723078 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.717351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43336 (local address 127.7.209.65:46847)
0929 03:05:45.718538 (+ 1187us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:45.718545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:45.718571 (+ 26us) server_negotiation.cc:407] Connection header received
0929 03:05:45.718655 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:45.718660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:45.718861 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:45.719010 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:45.719435 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.720218 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.721674 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.722356 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.722465 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.722642 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:45.722791 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 03:05:45.722916 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1068,"thread_start_us":72,"threads_started":1}
I20260929 03:05:45.723556 12583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.717218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:46103 (local address 127.0.0.1:36966)
0929 03:05:45.718362 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:45.718376 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:45.718468 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:45.719219 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:45.719222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:45.719240 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:45.719364 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.719371 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.720799 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.720802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.722101 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.722111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.723172 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.723176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.723271 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.723298 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 03:05:45.723343 (+ 45us) client_negotiation.cc:240] Negotiation successful
0929 03:05:45.723421 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1080,"mutex_wait_us":48,"thread_start_us":115,"threads_started":1}
I20260929 03:05:45.724342 12588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.717976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.7.209.67:46103)
0929 03:05:45.718743 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:45.718755 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:45.718777 (+ 22us) server_negotiation.cc:407] Connection header received
0929 03:05:45.718851 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:45.718856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:45.719024 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:45.719149 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:45.719499 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.720663 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.722259 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.723032 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.723896 (+ 864us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.724037 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:45.724089 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 03:05:45.724186 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":170,"threads_started":1}
I20260929 03:05:45.727208 12506 tablet_service.cc:1467] Processing CreateTablet for tablet c6e51d0ae0a7406a95b0d557d807cb6f (DEFAULT_TABLE table=test-table [id=5cd00e1b3ee94532b740095e5220b9af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:45.727212 12240 tablet_service.cc:1467] Processing CreateTablet for tablet c6e51d0ae0a7406a95b0d557d807cb6f (DEFAULT_TABLE table=test-table [id=5cd00e1b3ee94532b740095e5220b9af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:45.728317 12240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e51d0ae0a7406a95b0d557d807cb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:45.728324 12506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e51d0ae0a7406a95b0d557d807cb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:45.734025 12585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.716968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:37893 (local address 127.0.0.1:33076)
0929 03:05:45.718341 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:45.718354 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:45.718436 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:45.719009 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:45.719015 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:45.719038 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:45.719208 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.719215 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.720773 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.720776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.722101 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.722111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.723313 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.723316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.723389 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.723407 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 03:05:45.723451 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 03:05:45.723509 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":90,"thread_start_us":104,"threads_started":1}
I20260929 03:05:45.734031 12587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.717670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33076 (local address 127.7.209.66:37893)
0929 03:05:45.718220 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:45.718235 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:45.718544 (+ 309us) server_negotiation.cc:407] Connection header received
0929 03:05:45.718633 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:45.718638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:45.718850 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:45.719009 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:45.719349 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.720605 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.722232 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.723068 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.723175 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.723747 (+ 572us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:45.733724 (+ 9977us) server_negotiation.cc:299] Negotiation successful
0929 03:05:45.733864 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":139,"threads_started":1}
I20260929 03:05:45.735289 12589 tablet_bootstrap.cc:492] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea: Bootstrap starting.
I20260929 03:05:45.735664 12590 tablet_bootstrap.cc:492] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7: Bootstrap starting.
I20260929 03:05:45.737987 12590 tablet_bootstrap.cc:654] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:45.738020 12589 tablet_bootstrap.cc:654] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:45.737823 12373 tablet_service.cc:1467] Processing CreateTablet for tablet c6e51d0ae0a7406a95b0d557d807cb6f (DEFAULT_TABLE table=test-table [id=5cd00e1b3ee94532b740095e5220b9af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:45.738812 12590 log.cc:824] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:45.739156 12373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6e51d0ae0a7406a95b0d557d807cb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:45.739224 12589 log.cc:824] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:45.740880 12590 tablet_bootstrap.cc:492] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7: No bootstrap required, opened a new log
I20260929 03:05:45.741071 12590 ts_tablet_manager.cc:1397] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:05:45.741328 12589 tablet_bootstrap.cc:492] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea: No bootstrap required, opened a new log
I20260929 03:05:45.741521 12589 ts_tablet_manager.cc:1397] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260929 03:05:45.744702 12590 raft_consensus.cc:348] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae0485f9554c7b83b678cb412bf3a8" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37893 } } peers { permanent_uuid: "7532633e5c50435391a2943707ab1ca7" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 46847 } } peers { permanent_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 46103 } }
I20260929 03:05:45.744935 12590 raft_consensus.cc:374] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:45.744984 12590 raft_consensus.cc:729] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7532633e5c50435391a2943707ab1ca7, State: Initialized, Role: FOLLOWER
I20260929 03:05:45.745483 12590 consensus_queue.cc:260] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae0485f9554c7b83b678cb412bf3a8" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37893 } } peers { permanent_uuid: "7532633e5c50435391a2943707ab1ca7" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 46847 } } peers { permanent_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 46103 } }
I20260929 03:05:45.745410 12589 raft_consensus.cc:348] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae0485f9554c7b83b678cb412bf3a8" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37893 } } peers { permanent_uuid: "7532633e5c50435391a2943707ab1ca7" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 46847 } } peers { permanent_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 46103 } }
I20260929 03:05:45.745736 12589 raft_consensus.cc:374] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:45.745796 12589 raft_consensus.cc:729] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d44d16a48a74f7883c46cbcfbaf2fea, State: Initialized, Role: FOLLOWER
I20260929 03:05:45.746191 12306 heartbeater.cc:503] Master 127.7.209.126:41665 was elected leader, sending a full tablet report...
I20260929 03:05:45.746336 12589 consensus_queue.cc:260] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae0485f9554c7b83b678cb412bf3a8" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37893 } } peers { permanent_uuid: "7532633e5c50435391a2943707ab1ca7" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 46847 } } peers { permanent_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 46103 } }
I20260929 03:05:45.746582 12590 ts_tablet_manager.cc:1428] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 03:05:45.747320 12589 ts_tablet_manager.cc:1428] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 03:05:45.747691 12572 heartbeater.cc:503] Master 127.7.209.126:41665 was elected leader, sending a full tablet report...
I20260929 03:05:45.747741 12595 tablet_bootstrap.cc:492] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8: Bootstrap starting.
I20260929 03:05:45.750174 12595 tablet_bootstrap.cc:654] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:45.751282 12595 log.cc:824] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:45.753098 12595 tablet_bootstrap.cc:492] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8: No bootstrap required, opened a new log
I20260929 03:05:45.753278 12595 ts_tablet_manager.cc:1397] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:05:45.756242 12595 raft_consensus.cc:348] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae0485f9554c7b83b678cb412bf3a8" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37893 } } peers { permanent_uuid: "7532633e5c50435391a2943707ab1ca7" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 46847 } } peers { permanent_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 46103 } }
I20260929 03:05:45.756480 12595 raft_consensus.cc:374] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:45.756537 12595 raft_consensus.cc:729] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72ae0485f9554c7b83b678cb412bf3a8, State: Initialized, Role: FOLLOWER
I20260929 03:05:45.756950 12595 consensus_queue.cc:260] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae0485f9554c7b83b678cb412bf3a8" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37893 } } peers { permanent_uuid: "7532633e5c50435391a2943707ab1ca7" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 46847 } } peers { permanent_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 46103 } }
I20260929 03:05:45.757570 12439 heartbeater.cc:503] Master 127.7.209.126:41665 was elected leader, sending a full tablet report...
I20260929 03:05:45.757903 12595 ts_tablet_manager.cc:1428] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 03:05:45.828487 12597 raft_consensus.cc:482] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:45.828768 12597 raft_consensus.cc:504] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae0485f9554c7b83b678cb412bf3a8" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37893 } } peers { permanent_uuid: "7532633e5c50435391a2943707ab1ca7" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 46847 } } peers { permanent_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 46103 } }
I20260929 03:05:45.830595 12597 leader_election.cc:288] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7532633e5c50435391a2943707ab1ca7 (127.7.209.65:46847), 0d44d16a48a74f7883c46cbcfbaf2fea (127.7.209.67:46103)
I20260929 03:05:45.835297 12598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.830944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:46103 (local address 127.7.209.66:58579)
0929 03:05:45.831393 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:45.831404 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:45.831493 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:45.831761 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:45.831765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:45.831781 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:45.831900 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.831904 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.832629 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.832632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.834202 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.834212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.834860 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.834863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.834940 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.834962 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 03:05:45.835027 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 03:05:45.835111 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":94,"thread_start_us":190,"threads_started":1}
I20260929 03:05:45.835282 12588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.830984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:58579 (local address 127.7.209.67:46103)
0929 03:05:45.831109 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:45.831115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:45.831491 (+ 376us) server_negotiation.cc:407] Connection header received
0929 03:05:45.831611 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:45.831614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:45.831668 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:45.831738 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:45.832017 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.832530 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.834319 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.834762 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.834863 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.834916 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:45.835065 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 03:05:45.835168 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 03:05:45.835937 12441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.830891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:46847 (local address 127.7.209.66:56939)
0929 03:05:45.831107 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:45.831123 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:45.831234 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:45.831532 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:45.831536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:45.831551 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:45.831700 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.831706 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.832773 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.832777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.834202 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:45.834212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.835596 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.835599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:45.835694 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.835714 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 03:05:45.835768 (+ 54us) client_negotiation.cc:240] Negotiation successful
0929 03:05:45.835833 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":63}
I20260929 03:05:45.836174 12526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e51d0ae0a7406a95b0d557d807cb6f" candidate_uuid: "72ae0485f9554c7b83b678cb412bf3a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea" is_pre_election: true
I20260929 03:05:45.836601 12526 raft_consensus.cc:2454] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72ae0485f9554c7b83b678cb412bf3a8 in term 0.
I20260929 03:05:45.836716 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.831073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:56939 (local address 127.7.209.65:46847)
0929 03:05:45.831217 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:45.831222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:45.831240 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:05:45.831362 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:45.831365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:45.831425 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:45.831510 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:45.831837 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.832679 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.834980 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.835465 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.835591 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.835645 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:45.836475 (+ 830us) server_negotiation.cc:299] Negotiation successful
0929 03:05:45.836578 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 03:05:45.837183 12327 leader_election.cc:302] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d44d16a48a74f7883c46cbcfbaf2fea, 72ae0485f9554c7b83b678cb412bf3a8; no voters:
I20260929 03:05:45.837374 12260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e51d0ae0a7406a95b0d557d807cb6f" candidate_uuid: "72ae0485f9554c7b83b678cb412bf3a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7532633e5c50435391a2943707ab1ca7" is_pre_election: true
I20260929 03:05:45.837503 12597 raft_consensus.cc:2781] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:45.837597 12597 raft_consensus.cc:482] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:45.837671 12597 raft_consensus.cc:3037] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:45.837855 12260 raft_consensus.cc:2454] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72ae0485f9554c7b83b678cb412bf3a8 in term 0.
I20260929 03:05:45.839203 12597 raft_consensus.cc:504] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae0485f9554c7b83b678cb412bf3a8" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37893 } } peers { permanent_uuid: "7532633e5c50435391a2943707ab1ca7" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 46847 } } peers { permanent_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 46103 } }
I20260929 03:05:45.839569 12597 leader_election.cc:288] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [CANDIDATE]: Term 1 election: Requested vote from peers 7532633e5c50435391a2943707ab1ca7 (127.7.209.65:46847), 0d44d16a48a74f7883c46cbcfbaf2fea (127.7.209.67:46103)
I20260929 03:05:45.839931 12260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e51d0ae0a7406a95b0d557d807cb6f" candidate_uuid: "72ae0485f9554c7b83b678cb412bf3a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7532633e5c50435391a2943707ab1ca7"
I20260929 03:05:45.839989 12526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6e51d0ae0a7406a95b0d557d807cb6f" candidate_uuid: "72ae0485f9554c7b83b678cb412bf3a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea"
I20260929 03:05:45.840059 12260 raft_consensus.cc:3037] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:45.840132 12526 raft_consensus.cc:3037] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:45.841296 12260 raft_consensus.cc:2454] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72ae0485f9554c7b83b678cb412bf3a8 in term 1.
I20260929 03:05:45.841313 12526 raft_consensus.cc:2454] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72ae0485f9554c7b83b678cb412bf3a8 in term 1.
I20260929 03:05:45.841625 12328 leader_election.cc:302] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72ae0485f9554c7b83b678cb412bf3a8, 7532633e5c50435391a2943707ab1ca7; no voters:
I20260929 03:05:45.841936 12597 raft_consensus.cc:2781] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:45.842175 12597 raft_consensus.cc:686] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 1 LEADER]: Becoming Leader. State: Replica: 72ae0485f9554c7b83b678cb412bf3a8, State: Running, Role: LEADER
I20260929 03:05:45.842567 12597 consensus_queue.cc:237] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae0485f9554c7b83b678cb412bf3a8" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37893 } } peers { permanent_uuid: "7532633e5c50435391a2943707ab1ca7" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 46847 } } peers { permanent_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 46103 } }
I20260929 03:05:45.846212 12116 catalog_manager.cc:5495] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72ae0485f9554c7b83b678cb412bf3a8 (127.7.209.66). New cstate: current_term: 1 leader_uuid: "72ae0485f9554c7b83b678cb412bf3a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72ae0485f9554c7b83b678cb412bf3a8" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7532633e5c50435391a2943707ab1ca7" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 46847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 46103 } health_report { overall_health: UNKNOWN } } }
I20260929 03:05:45.877308 12587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.873191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.7.209.66:37893)
0929 03:05:45.873321 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:45.873324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:45.873338 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:05:45.873476 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:45.873479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:45.873537 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:45.873677 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:45.873947 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.874491 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.875705 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:45.875931 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:45.876007 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:45.876366 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:45.876890 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:45.876983 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:45.877070 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 03:05:45.877134 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:05:45.885061 12526 raft_consensus.cc:1261] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea [term 1 FOLLOWER]: Refusing update from remote peer 72ae0485f9554c7b83b678cb412bf3a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:45.885066 12260 raft_consensus.cc:1261] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7 [term 1 FOLLOWER]: Refusing update from remote peer 72ae0485f9554c7b83b678cb412bf3a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:45.885957 12599 consensus_queue.cc:1035] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d44d16a48a74f7883c46cbcfbaf2fea" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 46103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:45.886164 12597 consensus_queue.cc:1035] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7532633e5c50435391a2943707ab1ca7" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 46847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:45.893868 12602 mvcc.cc:205] Tried to move back new op lower bound from 7334507093536051200 to 7334507093374062592. Current Snapshot: MvccSnapshot[applied={T|T < 7334507093536051200}]
I20260929 03:05:45.895745 12603 mvcc.cc:205] Tried to move back new op lower bound from 7334507093536051200 to 7334507093374062592. Current Snapshot: MvccSnapshot[applied={T|T < 7334507093536051200}]
I20260929 03:05:45.904381 12116 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:45.904578 12116 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:45.907357 12116 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f02e8bbc64f4748b458a929d7286e81: Sending DeleteTablet for 3 replicas of tablet c6e51d0ae0a7406a95b0d557d807cb6f
I20260929 03:05:45.908260 12373 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e51d0ae0a7406a95b0d557d807cb6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:45 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:33076
I20260929 03:05:45.908282 12240 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e51d0ae0a7406a95b0d557d807cb6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:45 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:43336
I20260929 03:05:45.908281 12506 tablet_service.cc:1514] Processing DeleteTablet for tablet c6e51d0ae0a7406a95b0d557d807cb6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:45 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:36966
I20260929 03:05:45.908917 12613 tablet_replica.cc:330] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea: stopping tablet replica
I20260929 03:05:45.909083 12611 tablet_replica.cc:330] T c6e51d0ae0a7406a95b0d557d807cb6f P 7532633e5c50435391a2943707ab1ca7: stopping tablet replica
I20260929 03:05:45.909303 12612 tablet_replica.cc:330] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8: stopping tablet replica
I20260929 03:05:45.909397 12613 raft_consensus.cc:2229] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:45.909765 12612 raft_consensus.cc:2229] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:05:45.909893 12613 raft_consensus.cc:2258] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:45.910070 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 12178
I20260929 03:05:45.912707 12612 raft_consensus.cc:2258] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:45.912995 12613 ts_tablet_manager.cc:1905] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:45.914417 12612 ts_tablet_manager.cc:1905] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:45.916435 12613 ts_tablet_manager.cc:1918] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:45.916669 12613 log.cc:1196] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/c6e51d0ae0a7406a95b0d557d807cb6f
I20260929 03:05:45.917125 12613 ts_tablet_manager.cc:1939] T c6e51d0ae0a7406a95b0d557d807cb6f P 0d44d16a48a74f7883c46cbcfbaf2fea: Deleting consensus metadata
I20260929 03:05:45.917788 12612 ts_tablet_manager.cc:1918] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:45.917992 12612 log.cc:1196] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/c6e51d0ae0a7406a95b0d557d807cb6f
I20260929 03:05:45.918380 12612 ts_tablet_manager.cc:1939] T c6e51d0ae0a7406a95b0d557d807cb6f P 72ae0485f9554c7b83b678cb412bf3a8: Deleting consensus metadata
I20260929 03:05:45.919268 12101 catalog_manager.cc:4841] TS 0d44d16a48a74f7883c46cbcfbaf2fea (127.7.209.67:46103): tablet c6e51d0ae0a7406a95b0d557d807cb6f (table test-table [id=5cd00e1b3ee94532b740095e5220b9af]) successfully deleted
I20260929 03:05:45.919806 12103 catalog_manager.cc:4841] TS 72ae0485f9554c7b83b678cb412bf3a8 (127.7.209.66:37893): tablet c6e51d0ae0a7406a95b0d557d807cb6f (table test-table [id=5cd00e1b3ee94532b740095e5220b9af]) successfully deleted
W20260929 03:05:45.920702 12102 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:46847 (error 108)
I20260929 03:05:45.921012 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 12311
I20260929 03:05:45.922863 12585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:45.922482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:46847 (local address 127.0.0.1:43338)
0929 03:05:45.922643 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:45.922751 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 03:05:45.923074 12102 catalog_manager.cc:4584] TS 7532633e5c50435391a2943707ab1ca7 (127.7.209.65:46847): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c6e51d0ae0a7406a95b0d557d807cb6f: Network error: Client connection negotiation failed: client connection to 127.7.209.65:46847: connect: Connection refused (error 111)
I20260929 03:05:45.927507 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 12444
I20260929 03:05:45.933099 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 12085
2026-09-29T03:05:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2488 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T03:05:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:45Z Disabled control of system clock
I20260929 03:05:45.964322 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:46541
--webserver_interface=127.7.209.126
--webserver_port=0
--builtin_ntp_servers=127.7.209.84:34599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:46541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 03:05:46.089350 12618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:46.089774 12618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:46.089824 12618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:46.093906 12618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:46.093953 12618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:46.093966 12618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:46.093982 12618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:46.094002 12618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:46.098737 12618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:46541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:46541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:46.099961 12618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:46.101552 12618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:46.110471 12624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:46.110486 12623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:46.110623 12618 server_base.cc:1034] running on GCE node
W20260929 03:05:46.110486 12626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:46.111299 12618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:46.112460 12618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:46.113643 12618 hybrid_clock.cc:648] HybridClock initialized: now 1790651146113628 us; error 26 us; skew 500 ppm
I20260929 03:05:46.115648 12618 webserver.cc:460] Webserver started at http://127.7.209.126:43787/ using document root <none> and password file <none>
I20260929 03:05:46.116302 12618 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:46.116372 12618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:46.116590 12618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:46.118957 12618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "3816e39cdd3746c5b383ce58372aebd6"
format_stamp: "Formatted at 2026-09-29 03:05:46 on dist-test-slave-bdps"
server_key: "e1409a312c6bf857d6386d2a7b8f0374"
server_key_iv: "fc96912f77ce53d4543f629a2e81b04b"
server_key_version: "encryptionkey@0"
I20260929 03:05:46.119599 12618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "3816e39cdd3746c5b383ce58372aebd6"
format_stamp: "Formatted at 2026-09-29 03:05:46 on dist-test-slave-bdps"
server_key: "e1409a312c6bf857d6386d2a7b8f0374"
server_key_iv: "fc96912f77ce53d4543f629a2e81b04b"
server_key_version: "encryptionkey@0"
I20260929 03:05:46.123780 12618 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:05:46.126471 12632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:46.127782 12618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:46.127897 12618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "3816e39cdd3746c5b383ce58372aebd6"
format_stamp: "Formatted at 2026-09-29 03:05:46 on dist-test-slave-bdps"
server_key: "e1409a312c6bf857d6386d2a7b8f0374"
server_key_iv: "fc96912f77ce53d4543f629a2e81b04b"
server_key_version: "encryptionkey@0"
I20260929 03:05:46.127992 12618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:46.132839 12618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:46.133473 12618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:46.133662 12618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:46.140684 12618 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:46541
I20260929 03:05:46.140692 12684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:46541 every 8 connection(s)
I20260929 03:05:46.141629 12618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:46.145478 12685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:46.150743 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 12618
I20260929 03:05:46.150868 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:46.151142 8005 external_mini_cluster.cc:1428] Setting key cb6ab01b0641d27dfc12470051a5295e
I20260929 03:05:46.153024 12685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6: Bootstrap starting.
I20260929 03:05:46.155748 12685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:46.156617 12685 log.cc:824] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:46.157570 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.152778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52352 (local address 127.7.209.126:46541)
0929 03:05:46.153510 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:46.153528 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:46.153568 (+ 40us) server_negotiation.cc:407] Connection header received
0929 03:05:46.154349 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:46.154372 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:46.154803 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:46.155177 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:46.156114 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:46.156120 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:46.156133 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:46.156144 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:46.156212 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:46.156229 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:46.156332 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:46.156451 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 03:05:46.156752 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":146,"threads_started":1}
I20260929 03:05:46.159027 12685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6: No bootstrap required, opened a new log
I20260929 03:05:46.164083 12685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3816e39cdd3746c5b383ce58372aebd6" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46541 } }
I20260929 03:05:46.164381 12685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:46.164445 12685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3816e39cdd3746c5b383ce58372aebd6, State: Initialized, Role: FOLLOWER
I20260929 03:05:46.164973 12685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3816e39cdd3746c5b383ce58372aebd6" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46541 } }
I20260929 03:05:46.165165 12685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:46.165244 12685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:46.165354 12685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:46.166520 12685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3816e39cdd3746c5b383ce58372aebd6" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46541 } }
I20260929 03:05:46.166899 12685 leader_election.cc:302] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3816e39cdd3746c5b383ce58372aebd6; no voters:
I20260929 03:05:46.167248 12685 leader_election.cc:288] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:46.167356 12690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:46.167600 12690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [term 1 LEADER]: Becoming Leader. State: Replica: 3816e39cdd3746c5b383ce58372aebd6, State: Running, Role: LEADER
I20260929 03:05:46.167932 12690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3816e39cdd3746c5b383ce58372aebd6" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46541 } }
I20260929 03:05:46.168677 12685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:46.169781 12691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3816e39cdd3746c5b383ce58372aebd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3816e39cdd3746c5b383ce58372aebd6" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46541 } } }
I20260929 03:05:46.169919 12691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:46.170185 12692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3816e39cdd3746c5b383ce58372aebd6. Latest consensus state: current_term: 1 leader_uuid: "3816e39cdd3746c5b383ce58372aebd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3816e39cdd3746c5b383ce58372aebd6" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46541 } } }
I20260929 03:05:46.170274 12692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:46.171161 12699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:46.174266 12699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:46.180712 12699 catalog_manager.cc:1347] Generated new cluster ID: b891f1bf18b3483b94a2b4025c6c3066
I20260929 03:05:46.180799 12699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:46.192058 12699 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:46.192471 12699 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:46.198374 12699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6: Generated new TSK 0
I20260929 03:05:46.199287 12699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:05:46.203092 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:46541
--builtin_ntp_servers=127.7.209.84:34599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 03:05:46.330230 12709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:46.330540 12709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:46.330585 12709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:46.334774 12709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:46.334833 12709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:46.334905 12709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:46.339670 12709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:46541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:46.340968 12709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:46.342726 12709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:46.353485 12717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:46.353442 12714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:46.353464 12715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:46.353466 12709 server_base.cc:1034] running on GCE node
I20260929 03:05:46.354099 12709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:46.354785 12709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:46.355947 12709 hybrid_clock.cc:648] HybridClock initialized: now 1790651146355919 us; error 50 us; skew 500 ppm
I20260929 03:05:46.358062 12709 webserver.cc:460] Webserver started at http://127.7.209.65:42749/ using document root <none> and password file <none>
I20260929 03:05:46.358762 12709 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:46.358837 12709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:46.359089 12709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:46.361567 12709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "7033856634fe4ccb8e7d14345e92b44d"
format_stamp: "Formatted at 2026-09-29 03:05:46 on dist-test-slave-bdps"
server_key: "ea0c0063400a6b010f80cdcdedb05720"
server_key_iv: "98ca995a6224e92a04769efff5d367c1"
server_key_version: "encryptionkey@0"
I20260929 03:05:46.362259 12709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "7033856634fe4ccb8e7d14345e92b44d"
format_stamp: "Formatted at 2026-09-29 03:05:46 on dist-test-slave-bdps"
server_key: "ea0c0063400a6b010f80cdcdedb05720"
server_key_iv: "98ca995a6224e92a04769efff5d367c1"
server_key_version: "encryptionkey@0"
I20260929 03:05:46.366560 12709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:05:46.369345 12723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:46.370945 12709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 03:05:46.371061 12709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "7033856634fe4ccb8e7d14345e92b44d"
format_stamp: "Formatted at 2026-09-29 03:05:46 on dist-test-slave-bdps"
server_key: "ea0c0063400a6b010f80cdcdedb05720"
server_key_iv: "98ca995a6224e92a04769efff5d367c1"
server_key_version: "encryptionkey@0"
I20260929 03:05:46.371181 12709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:46.376371 12709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:46.377107 12709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:46.377275 12709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:46.377928 12709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:46.378964 12709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:46.379014 12709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:46.379068 12709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:46.379096 12709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:46.388619 12709 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:42371
I20260929 03:05:46.388640 12836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:42371 every 8 connection(s)
I20260929 03:05:46.389575 12709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:46.390970 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 12709
I20260929 03:05:46.391096 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:05:46.391407 8005 external_mini_cluster.cc:1428] Setting key c0262a496a20412b25aae7e7c79a7d0a
I20260929 03:05:46.393721 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:46541
--builtin_ntp_servers=127.7.209.84:34599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 03:05:46.396696 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.391680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:56327 (local address 127.7.209.126:46541)
0929 03:05:46.391852 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:46.391857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:46.393247 (+ 1390us) server_negotiation.cc:407] Connection header received
0929 03:05:46.394752 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:46.394756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:46.394800 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:46.394908 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:46.396257 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:46.396261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:46.396263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:46.396277 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:46.396303 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:46.396313 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:46.396396 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:46.396525 (+ 129us) server_negotiation.cc:299] Negotiation successful
0929 03:05:46.396570 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:05:46.398015 12839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.392224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:46541 (local address 127.7.209.65:56327)
0929 03:05:46.393084 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:46.393138 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:46.394536 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:46.395118 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:46.395126 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:46.395240 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:46.395912 (+ 672us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:05:46.395914 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:46.395977 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:05:46.396074 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:46.396389 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:46.396397 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:46.396402 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 03:05:46.396587 (+ 185us) client_negotiation.cc:240] Negotiation successful
0929 03:05:46.396880 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":144,"threads_started":1}
I20260929 03:05:46.399425 12837 heartbeater.cc:347] Connected to a master server at 127.7.209.126:46541
I20260929 03:05:46.399731 12837 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:46.400341 12837 heartbeater.cc:511] Master 127.7.209.126:46541 requested a full tablet report, sending...
I20260929 03:05:46.402442 12649 ts_manager.cc:194] Registered new tserver with Master: 7033856634fe4ccb8e7d14345e92b44d (127.7.209.65:42371)
W20260929 03:05:46.521842 12840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:46.522224 12840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:46.522294 12840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:46.526702 12840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:46.526775 12840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:46.526859 12840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:46.531883 12840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:46541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:46.533226 12840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:46.534909 12840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:46.545238 12846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:46.545354 12840 server_base.cc:1034] running on GCE node
W20260929 03:05:46.545269 12848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:46.545269 12845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:46.546015 12840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:46.546665 12840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:46.547817 12840 hybrid_clock.cc:648] HybridClock initialized: now 1790651146547801 us; error 30 us; skew 500 ppm
I20260929 03:05:46.550040 12840 webserver.cc:460] Webserver started at http://127.7.209.66:44425/ using document root <none> and password file <none>
I20260929 03:05:46.550675 12840 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:46.550742 12840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:46.550974 12840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:46.553326 12840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "7b52a76e81db468ab8ddd597666690da"
format_stamp: "Formatted at 2026-09-29 03:05:46 on dist-test-slave-bdps"
server_key: "4c811cb226e8ec84f5e1ebb1fdd2df41"
server_key_iv: "d546abec9c0a3a8d2c025fceba93c095"
server_key_version: "encryptionkey@0"
I20260929 03:05:46.553988 12840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "7b52a76e81db468ab8ddd597666690da"
format_stamp: "Formatted at 2026-09-29 03:05:46 on dist-test-slave-bdps"
server_key: "4c811cb226e8ec84f5e1ebb1fdd2df41"
server_key_iv: "d546abec9c0a3a8d2c025fceba93c095"
server_key_version: "encryptionkey@0"
I20260929 03:05:46.558446 12840 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 03:05:46.561477 12854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:46.563093 12840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260929 03:05:46.563241 12840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "7b52a76e81db468ab8ddd597666690da"
format_stamp: "Formatted at 2026-09-29 03:05:46 on dist-test-slave-bdps"
server_key: "4c811cb226e8ec84f5e1ebb1fdd2df41"
server_key_iv: "d546abec9c0a3a8d2c025fceba93c095"
server_key_version: "encryptionkey@0"
I20260929 03:05:46.563369 12840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:46.568707 12840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:46.569453 12840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:46.569669 12840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:46.570315 12840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:46.571388 12840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:46.571437 12840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:46.571491 12840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:46.571519 12840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:46.580982 12840 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:43825
I20260929 03:05:46.581022 12967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:43825 every 8 connection(s)
I20260929 03:05:46.582039 12840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:46.588680 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.584134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:48133 (local address 127.7.209.126:46541)
0929 03:05:46.584285 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:46.584289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:46.585572 (+ 1283us) server_negotiation.cc:407] Connection header received
0929 03:05:46.587049 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:46.587052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:46.587095 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:46.587166 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:46.588319 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:46.588321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:46.588323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:46.588331 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:46.588346 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:46.588353 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:46.588417 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:46.588511 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 03:05:46.588550 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 03:05:46.589700 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.584608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:46541 (local address 127.7.209.66:48133)
0929 03:05:46.585382 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:46.585445 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:46.586822 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:46.587340 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:46.587346 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:46.587452 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:46.588079 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:05:46.588080 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:46.588127 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:05:46.588203 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:46.588409 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:46.588414 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:46.588418 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 03:05:46.588547 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 03:05:46.588800 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":138,"threads_started":1}
I20260929 03:05:46.590883 12968 heartbeater.cc:347] Connected to a master server at 127.7.209.126:46541
I20260929 03:05:46.591117 12968 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:46.591564 12968 heartbeater.cc:511] Master 127.7.209.126:46541 requested a full tablet report, sending...
I20260929 03:05:46.592374 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 12840
I20260929 03:05:46.592515 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:05:46.592727 12649 ts_manager.cc:194] Registered new tserver with Master: 7b52a76e81db468ab8ddd597666690da (127.7.209.66:43825)
I20260929 03:05:46.592945 8005 external_mini_cluster.cc:1428] Setting key 66ab36980cc2c6aedfcbc19bd7f8f56b
I20260929 03:05:46.595041 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:46541
--builtin_ntp_servers=127.7.209.84:34599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 03:05:46.720745 12971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:46.721110 12971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:46.721179 12971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:46.725454 12971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:46.725519 12971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:46.725627 12971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:46.730676 12971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:46541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:46.731927 12971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:46.733589 12971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:46.743945 12979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:46.743952 12976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:46.743949 12977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:46.744087 12971 server_base.cc:1034] running on GCE node
I20260929 03:05:46.744539 12971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:46.745203 12971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:46.746368 12971 hybrid_clock.cc:648] HybridClock initialized: now 1790651146746348 us; error 31 us; skew 500 ppm
I20260929 03:05:46.748291 12971 webserver.cc:460] Webserver started at http://127.7.209.67:37127/ using document root <none> and password file <none>
I20260929 03:05:46.748914 12971 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:46.748983 12971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:46.749207 12971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:46.751616 12971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "ac1a0ab45b624198b4c5b22d7a49857f"
format_stamp: "Formatted at 2026-09-29 03:05:46 on dist-test-slave-bdps"
server_key: "9340d1234afa50f758b58d6ec3c209b9"
server_key_iv: "52001f921947b39da61a1b82c51abb7d"
server_key_version: "encryptionkey@0"
I20260929 03:05:46.752233 12971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "ac1a0ab45b624198b4c5b22d7a49857f"
format_stamp: "Formatted at 2026-09-29 03:05:46 on dist-test-slave-bdps"
server_key: "9340d1234afa50f758b58d6ec3c209b9"
server_key_iv: "52001f921947b39da61a1b82c51abb7d"
server_key_version: "encryptionkey@0"
I20260929 03:05:46.756275 12971 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260929 03:05:46.759178 12985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:46.760699 12971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:05:46.760833 12971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "ac1a0ab45b624198b4c5b22d7a49857f"
format_stamp: "Formatted at 2026-09-29 03:05:46 on dist-test-slave-bdps"
server_key: "9340d1234afa50f758b58d6ec3c209b9"
server_key_iv: "52001f921947b39da61a1b82c51abb7d"
server_key_version: "encryptionkey@0"
I20260929 03:05:46.760952 12971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:46.766304 12971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:46.767048 12971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:46.767218 12971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:46.767843 12971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:46.768976 12971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:46.769025 12971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:46.769080 12971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:46.769109 12971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:46.779206 12971 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:37199
I20260929 03:05:46.779230 13098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:37199 every 8 connection(s)
I20260929 03:05:46.780350 12971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:46.782445 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 12971
I20260929 03:05:46.782586 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:05:46.782939 8005 external_mini_cluster.cc:1428] Setting key b96afb0960d07add729fa744e9e82393
I20260929 03:05:46.787871 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.782610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:45673 (local address 127.7.209.126:46541)
0929 03:05:46.782802 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:46.782806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:46.784395 (+ 1589us) server_negotiation.cc:407] Connection header received
0929 03:05:46.785995 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:46.785999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:46.786052 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:46.786132 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:46.787377 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:46.787381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:46.787384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:46.787398 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:46.787425 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:46.787434 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:46.787516 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:46.787661 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 03:05:46.787713 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260929 03:05:46.788925 13101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.783302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:46541 (local address 127.7.209.67:45673)
0929 03:05:46.784204 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:46.784266 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:46.785764 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:46.786359 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:46.786367 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:46.786481 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:46.787077 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:05:46.787079 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:46.787128 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:05:46.787223 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:46.787505 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:46.787513 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:46.787519 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 03:05:46.787708 (+ 189us) client_negotiation.cc:240] Negotiation successful
0929 03:05:46.788004 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":141,"threads_started":1}
I20260929 03:05:46.790114 13099 heartbeater.cc:347] Connected to a master server at 127.7.209.126:46541
I20260929 03:05:46.790380 13099 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:46.790867 13099 heartbeater.cc:511] Master 127.7.209.126:46541 requested a full tablet report, sending...
I20260929 03:05:46.792009 12649 ts_manager.cc:194] Registered new tserver with Master: ac1a0ab45b624198b4c5b22d7a49857f (127.7.209.67:37199)
I20260929 03:05:46.797122 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:05:46.800989 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.799411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52354 (local address 127.7.209.126:46541)
0929 03:05:46.799550 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:46.799554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:46.799742 (+ 188us) server_negotiation.cc:407] Connection header received
0929 03:05:46.799852 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:46.799854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:46.799893 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:46.799961 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:46.800668 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:46.800671 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:46.800674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:46.800685 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:46.800706 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:46.800714 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:46.800780 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:46.800833 (+ 53us) server_negotiation.cc:299] Negotiation successful
0929 03:05:46.800870 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 03:05:46.804039 12649 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:05:46.807720 12649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:46.820555 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.816977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:37199 (local address 127.0.0.1:35946)
0929 03:05:46.817528 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:46.817585 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:46.817723 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:46.818559 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:46.818566 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:46.818671 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:46.819352 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:05:46.819354 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:46.819378 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:05:46.819454 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:46.820180 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:46.820190 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:46.820196 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 03:05:46.820312 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 03:05:46.820389 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":133,"threads_started":1}
I20260929 03:05:46.820839 13112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.817798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:43825 (local address 127.0.0.1:38836)
0929 03:05:46.818446 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:46.818462 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:46.818550 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:46.819357 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:46.819360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:46.819376 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:46.819843 (+ 467us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:05:46.819845 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:46.819856 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:05:46.819881 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:46.820588 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:46.820592 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:46.820596 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 03:05:46.820673 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 03:05:46.820710 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":94,"threads_started":1}
I20260929 03:05:46.820591 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.817180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35946 (local address 127.7.209.67:37199)
0929 03:05:46.818021 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:46.818034 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:46.818061 (+ 27us) server_negotiation.cc:407] Connection header received
0929 03:05:46.818125 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:46.818132 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:46.818370 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:46.819012 (+ 642us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:46.819978 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:46.819985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:46.819999 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:46.820010 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:46.820061 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:46.820105 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:46.820217 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:46.820339 (+ 122us) server_negotiation.cc:299] Negotiation successful
0929 03:05:46.820426 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":123,"threads_started":1}
I20260929 03:05:46.821305 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.818045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38836 (local address 127.7.209.66:43825)
0929 03:05:46.818483 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:46.818499 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:46.818528 (+ 29us) server_negotiation.cc:407] Connection header received
0929 03:05:46.818709 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:46.818716 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:46.818901 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:46.819044 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:46.820410 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:46.820415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:46.820425 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:46.820433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:46.820468 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:46.820484 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:46.820586 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:46.820905 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 03:05:46.821123 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":109,"threads_started":1}
I20260929 03:05:46.821394 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.818122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:42371 (local address 127.0.0.1:56882)
0929 03:05:46.818655 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:46.818668 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:46.818736 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:46.819969 (+ 1233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:46.819972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:46.819986 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:46.820745 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:05:46.820747 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:46.820756 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:05:46.820780 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:46.821170 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:46.821173 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:46.821176 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 03:05:46.821237 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 03:05:46.821272 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":87,"threads_started":1}
I20260929 03:05:46.821852 13115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.818343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56882 (local address 127.7.209.65:42371)
0929 03:05:46.818850 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:46.818862 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:46.818887 (+ 25us) server_negotiation.cc:407] Connection header received
0929 03:05:46.818948 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:46.818954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:46.819132 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:46.819286 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:46.820917 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:46.820923 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:46.820934 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:46.820946 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:46.820994 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:46.821027 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:46.821118 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:46.821565 (+ 447us) server_negotiation.cc:299] Negotiation successful
0929 03:05:46.821669 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":175,"threads_started":1}
I20260929 03:05:46.824872 12771 tablet_service.cc:1467] Processing CreateTablet for tablet fbfe5c7a0bb5403fa83323ecb279edba (DEFAULT_TABLE table=test-table [id=001ba97748cf4f879fe747ab290c8b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:46.824872 12902 tablet_service.cc:1467] Processing CreateTablet for tablet fbfe5c7a0bb5403fa83323ecb279edba (DEFAULT_TABLE table=test-table [id=001ba97748cf4f879fe747ab290c8b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:46.824870 13033 tablet_service.cc:1467] Processing CreateTablet for tablet fbfe5c7a0bb5403fa83323ecb279edba (DEFAULT_TABLE table=test-table [id=001ba97748cf4f879fe747ab290c8b4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:46.825995 12771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbfe5c7a0bb5403fa83323ecb279edba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:46.825999 12902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbfe5c7a0bb5403fa83323ecb279edba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:46.825999 13033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbfe5c7a0bb5403fa83323ecb279edba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:46.832754 13116 tablet_bootstrap.cc:492] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d: Bootstrap starting.
I20260929 03:05:46.834261 13117 tablet_bootstrap.cc:492] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da: Bootstrap starting.
I20260929 03:05:46.834283 13118 tablet_bootstrap.cc:492] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f: Bootstrap starting.
I20260929 03:05:46.835913 13116 tablet_bootstrap.cc:654] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:46.836196 13117 tablet_bootstrap.cc:654] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:46.836311 13118 tablet_bootstrap.cc:654] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:46.836915 13117 log.cc:824] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:46.836922 13116 log.cc:824] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:46.837056 13118 log.cc:824] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:46.839053 13118 tablet_bootstrap.cc:492] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f: No bootstrap required, opened a new log
I20260929 03:05:46.839053 13117 tablet_bootstrap.cc:492] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da: No bootstrap required, opened a new log
I20260929 03:05:46.839313 13118 ts_tablet_manager.cc:1397] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 03:05:46.839313 13117 ts_tablet_manager.cc:1397] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 03:05:46.839891 13116 tablet_bootstrap.cc:492] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d: No bootstrap required, opened a new log
I20260929 03:05:46.840077 13116 ts_tablet_manager.cc:1397] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260929 03:05:46.843255 13117 raft_consensus.cc:348] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37199 } } peers { permanent_uuid: "7b52a76e81db468ab8ddd597666690da" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 43825 } } peers { permanent_uuid: "7033856634fe4ccb8e7d14345e92b44d" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 42371 } }
I20260929 03:05:46.843513 13117 raft_consensus.cc:374] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:46.843568 13117 raft_consensus.cc:729] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b52a76e81db468ab8ddd597666690da, State: Initialized, Role: FOLLOWER
I20260929 03:05:46.844053 13117 consensus_queue.cc:260] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37199 } } peers { permanent_uuid: "7b52a76e81db468ab8ddd597666690da" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 43825 } } peers { permanent_uuid: "7033856634fe4ccb8e7d14345e92b44d" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 42371 } }
I20260929 03:05:46.843969 13118 raft_consensus.cc:348] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37199 } } peers { permanent_uuid: "7b52a76e81db468ab8ddd597666690da" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 43825 } } peers { permanent_uuid: "7033856634fe4ccb8e7d14345e92b44d" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 42371 } }
I20260929 03:05:46.844236 13118 raft_consensus.cc:374] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:46.844295 13118 raft_consensus.cc:729] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1a0ab45b624198b4c5b22d7a49857f, State: Initialized, Role: FOLLOWER
I20260929 03:05:46.844527 13116 raft_consensus.cc:348] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37199 } } peers { permanent_uuid: "7b52a76e81db468ab8ddd597666690da" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 43825 } } peers { permanent_uuid: "7033856634fe4ccb8e7d14345e92b44d" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 42371 } }
I20260929 03:05:46.844764 13116 raft_consensus.cc:374] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:46.844807 13116 raft_consensus.cc:729] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7033856634fe4ccb8e7d14345e92b44d, State: Initialized, Role: FOLLOWER
I20260929 03:05:46.844795 13118 consensus_queue.cc:260] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37199 } } peers { permanent_uuid: "7b52a76e81db468ab8ddd597666690da" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 43825 } } peers { permanent_uuid: "7033856634fe4ccb8e7d14345e92b44d" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 42371 } }
I20260929 03:05:46.845252 13117 ts_tablet_manager.cc:1428] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 03:05:46.845283 13116 consensus_queue.cc:260] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37199 } } peers { permanent_uuid: "7b52a76e81db468ab8ddd597666690da" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 43825 } } peers { permanent_uuid: "7033856634fe4ccb8e7d14345e92b44d" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 42371 } }
I20260929 03:05:46.845676 12968 heartbeater.cc:503] Master 127.7.209.126:46541 was elected leader, sending a full tablet report...
I20260929 03:05:46.845831 13118 ts_tablet_manager.cc:1428] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260929 03:05:46.845979 13099 heartbeater.cc:503] Master 127.7.209.126:46541 was elected leader, sending a full tablet report...
I20260929 03:05:46.846208 13116 ts_tablet_manager.cc:1428] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 03:05:46.846515 12837 heartbeater.cc:503] Master 127.7.209.126:46541 was elected leader, sending a full tablet report...
W20260929 03:05:46.890818 12838 tablet.cc:2369] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:05:46.948196 13123 raft_consensus.cc:482] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:46.948446 13123 raft_consensus.cc:504] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37199 } } peers { permanent_uuid: "7b52a76e81db468ab8ddd597666690da" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 43825 } } peers { permanent_uuid: "7033856634fe4ccb8e7d14345e92b44d" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 42371 } }
I20260929 03:05:46.949952 13123 leader_election.cc:288] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b52a76e81db468ab8ddd597666690da (127.7.209.66:43825), 7033856634fe4ccb8e7d14345e92b44d (127.7.209.65:42371)
I20260929 03:05:46.952003 13101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.950192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:43825 (local address 127.7.209.67:54299)
0929 03:05:46.950340 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:46.950355 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:46.950446 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:46.950817 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:46.950820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:46.950839 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:46.951418 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:05:46.951420 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:46.951435 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:05:46.951463 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:46.951740 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:46.951744 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:46.951747 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 03:05:46.951826 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 03:05:46.951864 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260929 03:05:46.952026 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.950344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:54299 (local address 127.7.209.66:43825)
0929 03:05:46.950506 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:46.950511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:46.950528 (+ 17us) server_negotiation.cc:407] Connection header received
0929 03:05:46.950577 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:46.950579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:46.950616 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:46.950705 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:46.951598 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:46.951603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:46.951605 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:46.951620 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:46.951647 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:46.951660 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:46.951749 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:46.951834 (+ 85us) server_negotiation.cc:299] Negotiation successful
0929 03:05:46.951880 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:05:46.952184 13115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.950752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:54853 (local address 127.7.209.65:42371)
0929 03:05:46.950883 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:46.950888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:46.951074 (+ 186us) server_negotiation.cc:407] Connection header received
0929 03:05:46.951174 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:46.951177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:46.951218 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:46.951299 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:46.951826 (+ 527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:46.951829 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:46.951832 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:46.951843 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:46.951862 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:46.951871 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:46.951953 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:46.952017 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 03:05:46.952055 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 03:05:46.952199 13125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.950483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:42371 (local address 127.7.209.67:54853)
0929 03:05:46.950977 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:46.950990 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:46.951088 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:46.951321 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:46.951323 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:46.951336 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:46.951693 (+ 357us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:05:46.951694 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:46.951703 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:05:46.951730 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:46.951945 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:46.951948 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:46.951951 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 03:05:46.952020 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 03:05:46.952052 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":144,"threads_started":1}
I20260929 03:05:46.952901 12922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbfe5c7a0bb5403fa83323ecb279edba" candidate_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b52a76e81db468ab8ddd597666690da" is_pre_election: true
I20260929 03:05:46.952980 12791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbfe5c7a0bb5403fa83323ecb279edba" candidate_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7033856634fe4ccb8e7d14345e92b44d" is_pre_election: true
I20260929 03:05:46.953284 12922 raft_consensus.cc:2454] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac1a0ab45b624198b4c5b22d7a49857f in term 0.
I20260929 03:05:46.953284 12791 raft_consensus.cc:2454] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac1a0ab45b624198b4c5b22d7a49857f in term 0.
I20260929 03:05:46.953800 12988 leader_election.cc:302] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b52a76e81db468ab8ddd597666690da, ac1a0ab45b624198b4c5b22d7a49857f; no voters:
I20260929 03:05:46.954058 13123 raft_consensus.cc:2781] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:46.954157 13123 raft_consensus.cc:482] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:46.954207 13123 raft_consensus.cc:3037] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:46.955440 13123 raft_consensus.cc:504] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37199 } } peers { permanent_uuid: "7b52a76e81db468ab8ddd597666690da" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 43825 } } peers { permanent_uuid: "7033856634fe4ccb8e7d14345e92b44d" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 42371 } }
I20260929 03:05:46.955772 13123 leader_election.cc:288] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [CANDIDATE]: Term 1 election: Requested vote from peers 7b52a76e81db468ab8ddd597666690da (127.7.209.66:43825), 7033856634fe4ccb8e7d14345e92b44d (127.7.209.65:42371)
I20260929 03:05:46.956085 12922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbfe5c7a0bb5403fa83323ecb279edba" candidate_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b52a76e81db468ab8ddd597666690da"
I20260929 03:05:46.956167 12791 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbfe5c7a0bb5403fa83323ecb279edba" candidate_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7033856634fe4ccb8e7d14345e92b44d"
I20260929 03:05:46.956231 12922 raft_consensus.cc:3037] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:46.956290 12791 raft_consensus.cc:3037] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:46.957818 12922 raft_consensus.cc:2454] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac1a0ab45b624198b4c5b22d7a49857f in term 1.
I20260929 03:05:46.957962 12791 raft_consensus.cc:2454] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac1a0ab45b624198b4c5b22d7a49857f in term 1.
I20260929 03:05:46.958163 12988 leader_election.cc:302] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b52a76e81db468ab8ddd597666690da, ac1a0ab45b624198b4c5b22d7a49857f; no voters:
I20260929 03:05:46.958338 13123 raft_consensus.cc:2781] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:46.958592 13123 raft_consensus.cc:686] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 1 LEADER]: Becoming Leader. State: Replica: ac1a0ab45b624198b4c5b22d7a49857f, State: Running, Role: LEADER
I20260929 03:05:46.959036 13123 consensus_queue.cc:237] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37199 } } peers { permanent_uuid: "7b52a76e81db468ab8ddd597666690da" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 43825 } } peers { permanent_uuid: "7033856634fe4ccb8e7d14345e92b44d" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 42371 } }
I20260929 03:05:46.962903 12648 catalog_manager.cc:5495] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f reported cstate change: term changed from 0 to 1, leader changed from <none> to ac1a0ab45b624198b4c5b22d7a49857f (127.7.209.67). New cstate: current_term: 1 leader_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac1a0ab45b624198b4c5b22d7a49857f" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b52a76e81db468ab8ddd597666690da" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 43825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7033856634fe4ccb8e7d14345e92b44d" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 42371 } health_report { overall_health: UNKNOWN } } }
I20260929 03:05:46.979866 13111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:46.978386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35948 (local address 127.7.209.67:37199)
0929 03:05:46.978518 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:46.978522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:46.978535 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:05:46.978674 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:46.978677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:46.978748 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:46.978821 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:46.979520 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:46.979522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:46.979524 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:46.979531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:46.979548 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:46.979555 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:46.979611 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:46.979670 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 03:05:46.979705 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:05:46.987002 12791 raft_consensus.cc:1261] T fbfe5c7a0bb5403fa83323ecb279edba P 7033856634fe4ccb8e7d14345e92b44d [term 1 FOLLOWER]: Refusing update from remote peer ac1a0ab45b624198b4c5b22d7a49857f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:46.987003 12922 raft_consensus.cc:1261] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da [term 1 FOLLOWER]: Refusing update from remote peer ac1a0ab45b624198b4c5b22d7a49857f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:46.987757 13126 consensus_queue.cc:1035] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7033856634fe4ccb8e7d14345e92b44d" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 42371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:46.987936 13123 consensus_queue.cc:1035] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b52a76e81db468ab8ddd597666690da" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 43825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:46.995748 13130 mvcc.cc:205] Tried to move back new op lower bound from 7334507098050170880 to 7334507097947471872. Current Snapshot: MvccSnapshot[applied={T|T < 7334507098050170880}]
I20260929 03:05:47.005208 12647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:47.005406 12647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:47.008002 12647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3816e39cdd3746c5b383ce58372aebd6: Sending DeleteTablet for 3 replicas of tablet fbfe5c7a0bb5403fa83323ecb279edba
I20260929 03:05:47.008868 12902 tablet_service.cc:1514] Processing DeleteTablet for tablet fbfe5c7a0bb5403fa83323ecb279edba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:47 UTC) from {username='slave'} at 127.0.0.1:38836
I20260929 03:05:47.008939 12771 tablet_service.cc:1514] Processing DeleteTablet for tablet fbfe5c7a0bb5403fa83323ecb279edba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:47 UTC) from {username='slave'} at 127.0.0.1:56882
I20260929 03:05:47.009418 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 12709
I20260929 03:05:47.009629 13033 tablet_service.cc:1514] Processing DeleteTablet for tablet fbfe5c7a0bb5403fa83323ecb279edba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:47 UTC) from {username='slave'} at 127.0.0.1:35946
I20260929 03:05:47.010340 13140 tablet_replica.cc:330] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f: stopping tablet replica
I20260929 03:05:47.011117 13138 tablet_replica.cc:330] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da: stopping tablet replica
I20260929 03:05:47.011291 13140 raft_consensus.cc:2229] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:05:47.011564 13138 raft_consensus.cc:2229] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:47.011686 13140 raft_consensus.cc:2258] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:47.011997 13138 raft_consensus.cc:2258] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:47.013200 13140 ts_tablet_manager.cc:1905] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:47.013864 13138 ts_tablet_manager.cc:1905] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:47.016476 13140 ts_tablet_manager.cc:1918] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:47.016626 13140 log.cc:1196] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/fbfe5c7a0bb5403fa83323ecb279edba
I20260929 03:05:47.016848 13138 ts_tablet_manager.cc:1918] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:47.017004 13140 ts_tablet_manager.cc:1939] T fbfe5c7a0bb5403fa83323ecb279edba P ac1a0ab45b624198b4c5b22d7a49857f: Deleting consensus metadata
I20260929 03:05:47.017014 13138 log.cc:1196] T fbfe5c7a0bb5403fa83323ecb279edba P 7b52a76e81db468ab8ddd597666690da: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/fbfe5c7a0bb5403fa83323ecb279edba
W20260929 03:05:47.017411 12635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:42371 (error 108)
I20260929 03:05:47.017633 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 12840
I20260929 03:05:47.019351 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:47.018990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:42371 (local address 127.0.0.1:56898)
0929 03:05:47.019176 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:47.019249 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:42371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260929 03:05:47.019619 12635 catalog_manager.cc:4584] TS 7033856634fe4ccb8e7d14345e92b44d (127.7.209.65:42371): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fbfe5c7a0bb5403fa83323ecb279edba: Network error: Client connection negotiation failed: client connection to 127.7.209.65:42371: connect: Connection refused (error 111)
I20260929 03:05:47.020104 12635 catalog_manager.cc:4841] TS ac1a0ab45b624198b4c5b22d7a49857f (127.7.209.67:37199): tablet fbfe5c7a0bb5403fa83323ecb279edba (table test-table [id=001ba97748cf4f879fe747ab290c8b4f]) successfully deleted
I20260929 03:05:47.023274 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 12971
I20260929 03:05:47.023407 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:47.023165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:43825 (local address 127.0.0.1:38838)
0929 03:05:47.023278 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:47.023332 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.66:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 03:05:47.028304 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 12618
2026-09-29T03:05:47Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1094 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 03:05:47.049883 8005 mini_oidc.cc:152] Starting JWKS server
I20260929 03:05:47.051816 8005 webserver.cc:460] Webserver started at https://127.0.0.1:41261/ using document root <none> and password file <none>
I20260929 03:05:47.051996 8005 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 03:05:47.052278 8005 webserver.cc:460] Webserver started at http://127.0.0.1:37159/ using document root <none> and password file <none>
2026-09-29T03:05:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:47Z Disabled control of system clock
I20260929 03:05:47.062500 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:45537
--webserver_interface=127.7.209.126
--webserver_port=0
--builtin_ntp_servers=127.7.209.84:38607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:45537
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41261/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/testchainca.cert with env {}
W20260929 03:05:47.187161 13147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:47.187439 13147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:47.187479 13147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:47.191586 13147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:47.191648 13147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:47.191663 13147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:47.191679 13147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:47.191694 13147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 03:05:47.191710 13147 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 03:05:47.191737 13147 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41261/jwks.json
I20260929 03:05:47.197438 13147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:45537
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:45537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41261/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:47.198879 13147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:47.200233 13147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:47.209985 13153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:47.210005 13152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:47.209986 13155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:47.210173 13147 server_base.cc:1034] running on GCE node
I20260929 03:05:47.210873 13147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:47.212141 13147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:47.213289 13147 hybrid_clock.cc:648] HybridClock initialized: now 1790651147213259 us; error 46 us; skew 500 ppm
I20260929 03:05:47.215435 13147 webserver.cc:460] Webserver started at http://127.7.209.126:36535/ using document root <none> and password file <none>
I20260929 03:05:47.216089 13147 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:47.216154 13147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:47.216379 13147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:47.219062 13147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "72c5d718eacf4ac98ddb15910bc8b711"
format_stamp: "Formatted at 2026-09-29 03:05:47 on dist-test-slave-bdps"
server_key: "93c5d84e7bc6aa540233b8c2ac7b9697"
server_key_iv: "71003ef92f82bc1d31736cdd886f366c"
server_key_version: "encryptionkey@0"
I20260929 03:05:47.219729 13147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "72c5d718eacf4ac98ddb15910bc8b711"
format_stamp: "Formatted at 2026-09-29 03:05:47 on dist-test-slave-bdps"
server_key: "93c5d84e7bc6aa540233b8c2ac7b9697"
server_key_iv: "71003ef92f82bc1d31736cdd886f366c"
server_key_version: "encryptionkey@0"
I20260929 03:05:47.223925 13147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260929 03:05:47.226557 13161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:47.227988 13147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:47.228107 13147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "72c5d718eacf4ac98ddb15910bc8b711"
format_stamp: "Formatted at 2026-09-29 03:05:47 on dist-test-slave-bdps"
server_key: "93c5d84e7bc6aa540233b8c2ac7b9697"
server_key_iv: "71003ef92f82bc1d31736cdd886f366c"
server_key_version: "encryptionkey@0"
I20260929 03:05:47.228205 13147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:47.251523 13147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:47.252482 13147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:47.252699 13147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:47.260615 13147 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:45537
I20260929 03:05:47.260612 13213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:45537 every 8 connection(s)
I20260929 03:05:47.261792 13147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:47.265532 13214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:47.268715 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 13147
I20260929 03:05:47.268857 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:47.269225 8005 external_mini_cluster.cc:1428] Setting key b9eff26451ec807e281992e88651bcbd
I20260929 03:05:47.272864 13214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711: Bootstrap starting.
I20260929 03:05:47.276054 13214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:47.276994 13214 log.cc:824] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:47.279850 13214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711: No bootstrap required, opened a new log
I20260929 03:05:47.280441 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:47.270902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46576 (local address 127.7.209.126:45537)
0929 03:05:47.271661 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:47.271678 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:47.271724 (+ 46us) server_negotiation.cc:407] Connection header received
0929 03:05:47.272480 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:47.272501 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:47.273034 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:47.273398 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:47.273870 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.274963 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.276234 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.276799 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.276923 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:47.277866 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:47.277872 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:47.277887 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:47.278483 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:47.278575 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:47.278605 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:47.278710 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:47.278888 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 03:05:47.279235 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":154,"threads_started":1}
I20260929 03:05:47.283921 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:05:47.284767 13214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c5d718eacf4ac98ddb15910bc8b711" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45537 } }
I20260929 03:05:47.285106 13214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:47.285172 13214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c5d718eacf4ac98ddb15910bc8b711, State: Initialized, Role: FOLLOWER
I20260929 03:05:47.285753 13214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c5d718eacf4ac98ddb15910bc8b711" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45537 } }
I20260929 03:05:47.285929 13214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:47.286000 13214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:47.286108 13214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:47.287212 13214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c5d718eacf4ac98ddb15910bc8b711" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45537 } }
I20260929 03:05:47.287616 13214 leader_election.cc:302] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72c5d718eacf4ac98ddb15910bc8b711; no voters:
I20260929 03:05:47.287961 13214 leader_election.cc:288] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:47.288066 13219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:47.288280 13219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [term 1 LEADER]: Becoming Leader. State: Replica: 72c5d718eacf4ac98ddb15910bc8b711, State: Running, Role: LEADER
I20260929 03:05:47.288616 13219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c5d718eacf4ac98ddb15910bc8b711" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45537 } }
I20260929 03:05:47.289324 13214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:47.290408 13220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72c5d718eacf4ac98ddb15910bc8b711" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c5d718eacf4ac98ddb15910bc8b711" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45537 } } }
I20260929 03:05:47.290560 13220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:47.290395 13221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72c5d718eacf4ac98ddb15910bc8b711. Latest consensus state: current_term: 1 leader_uuid: "72c5d718eacf4ac98ddb15910bc8b711" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c5d718eacf4ac98ddb15910bc8b711" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45537 } } }
I20260929 03:05:47.290684 13221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:47.290949 13228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:47.294653 13228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:47.301537 13228 catalog_manager.cc:1347] Generated new cluster ID: 2f54c9b1c0524e7d9dbb0eb262c4d00a
I20260929 03:05:47.301622 13228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:47.327785 13228 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:47.328919 13228 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:47.340065 13228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72c5d718eacf4ac98ddb15910bc8b711: Generated new TSK 0
I20260929 03:05:47.341073 13228 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 03:05:47.378265 13244 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 03:05:47.378422 13244 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 03:05:47.379004 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:47.361410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.7.209.126:45537)
0929 03:05:47.361568 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:47.361573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:47.361850 (+ 277us) server_negotiation.cc:407] Connection header received
0929 03:05:47.362116 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:47.362118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:47.362174 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:47.362273 (+ 99us) server_negotiation.cc:226] Negotiated authn=JWT
0929 03:05:47.362688 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.363271 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.364620 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.364876 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.364956 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:47.365381 (+ 425us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 03:05:47.378537 (+ 13156us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41261/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 03:05:47.378848 (+ 311us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40094: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41261/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260929 03:05:47.379099 13217 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40094: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41261/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260929 03:05:47.380782 13243 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:05:47.361312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:45537 (local address 127.0.0.1:40094)
0929 03:05:47.361744 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:47.361763 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:47.361935 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:47.362307 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:47.362310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:47.362320 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0929 03:05:47.362484 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:47.362491 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.363408 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.363412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:47.364484 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:47.364495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.364989 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.364994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:47.365110 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:47.365182 (+ 72us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 03:05:47.378839 (+ 13657us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41261/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 03:05:47.379173 (+ 334us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.209.126:45537: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41261/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":109,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.209.126:45537: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41261/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 03:05:47.381781 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 13147
2026-09-29T03:05:47Z chronyd exiting
I20260929 03:05:47.420228 8005 test_util.cc:175] -----------------------------------------------
I20260929 03:05:47.420349 8005 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790651120485886-8005-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (372 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 03:05:47.421567 8005 mini_oidc.cc:152] Starting JWKS server
I20260929 03:05:47.422602 8005 webserver.cc:460] Webserver started at https://127.0.0.1:37195/ using document root <none> and password file <none>
I20260929 03:05:47.422698 8005 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 03:05:47.422973 8005 webserver.cc:460] Webserver started at http://127.0.0.1:38749/ using document root <none> and password file <none>
2026-09-29T03:05:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:47Z Disabled control of system clock
I20260929 03:05:47.433768 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:45315
--webserver_interface=127.7.209.126
--webserver_port=0
--builtin_ntp_servers=127.7.209.84:36653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:45315
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37195/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/testchainca.cert with env {}
W20260929 03:05:47.560330 13250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:47.560729 13250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:47.560775 13250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:47.565011 13250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:47.565088 13250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:47.565102 13250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:47.565119 13250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:47.565132 13250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 03:05:47.565148 13250 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 03:05:47.565160 13250 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37195/jwks.json
I20260929 03:05:47.571112 13250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:36653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:45315
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:45315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37195/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:47.572487 13250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:47.573725 13250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:47.583328 13258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:47.583389 13255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:47.583402 13256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:47.583521 13250 server_base.cc:1034] running on GCE node
I20260929 03:05:47.584208 13250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:47.585484 13250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:47.586671 13250 hybrid_clock.cc:648] HybridClock initialized: now 1790651147586651 us; error 33 us; skew 500 ppm
I20260929 03:05:47.588784 13250 webserver.cc:460] Webserver started at http://127.7.209.126:44657/ using document root <none> and password file <none>
I20260929 03:05:47.589459 13250 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:47.589529 13250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:47.589813 13250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:47.592190 13250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "28c277ed6c484dabacc1a59b037e8af5"
format_stamp: "Formatted at 2026-09-29 03:05:47 on dist-test-slave-bdps"
server_key: "faaf2a683e2cab7f61757c571f76564e"
server_key_iv: "e555226bd44ab5c4aa749604ac0a98b2"
server_key_version: "encryptionkey@0"
I20260929 03:05:47.592929 13250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "28c277ed6c484dabacc1a59b037e8af5"
format_stamp: "Formatted at 2026-09-29 03:05:47 on dist-test-slave-bdps"
server_key: "faaf2a683e2cab7f61757c571f76564e"
server_key_iv: "e555226bd44ab5c4aa749604ac0a98b2"
server_key_version: "encryptionkey@0"
I20260929 03:05:47.597316 13250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 03:05:47.600346 13264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:47.601955 13250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:47.602082 13250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "28c277ed6c484dabacc1a59b037e8af5"
format_stamp: "Formatted at 2026-09-29 03:05:47 on dist-test-slave-bdps"
server_key: "faaf2a683e2cab7f61757c571f76564e"
server_key_iv: "e555226bd44ab5c4aa749604ac0a98b2"
server_key_version: "encryptionkey@0"
I20260929 03:05:47.602205 13250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:47.628123 13250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:47.629024 13250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:47.629242 13250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:47.637439 13250 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:45315
I20260929 03:05:47.637455 13316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:45315 every 8 connection(s)
I20260929 03:05:47.638733 13250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:47.640980 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 13250
I20260929 03:05:47.641106 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:47.641434 8005 external_mini_cluster.cc:1428] Setting key d0850042140681554b5f567d355c7c64
I20260929 03:05:47.643056 13317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:47.649710 13317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5: Bootstrap starting.
I20260929 03:05:47.651695 13320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:47.643082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46108 (local address 127.7.209.126:45315)
0929 03:05:47.643847 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:47.643866 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:47.643910 (+ 44us) server_negotiation.cc:407] Connection header received
0929 03:05:47.644683 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:47.644702 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:47.645167 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:47.645527 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:47.646044 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.647128 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.648102 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.648616 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.648780 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:47.649789 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:47.649794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:47.649809 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:47.650270 (+ 461us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:47.650333 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:47.650357 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:47.650446 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:47.650593 (+ 147us) server_negotiation.cc:299] Negotiation successful
0929 03:05:47.650874 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":167,"threads_started":1}
I20260929 03:05:47.652345 13317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:47.653209 13317 log.cc:824] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:47.654666 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:05:47.655411 13317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5: No bootstrap required, opened a new log
I20260929 03:05:47.659538 13317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c277ed6c484dabacc1a59b037e8af5" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45315 } }
I20260929 03:05:47.659833 13317 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:47.659904 13317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c277ed6c484dabacc1a59b037e8af5, State: Initialized, Role: FOLLOWER
I20260929 03:05:47.660391 13317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c277ed6c484dabacc1a59b037e8af5" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45315 } }
I20260929 03:05:47.660531 13317 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:47.660599 13317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:47.660712 13317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:47.661749 13317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c277ed6c484dabacc1a59b037e8af5" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45315 } }
I20260929 03:05:47.662109 13317 leader_election.cc:302] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28c277ed6c484dabacc1a59b037e8af5; no voters:
I20260929 03:05:47.662485 13317 leader_election.cc:288] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:47.662590 13322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:47.662798 13322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [term 1 LEADER]: Becoming Leader. State: Replica: 28c277ed6c484dabacc1a59b037e8af5, State: Running, Role: LEADER
I20260929 03:05:47.663106 13322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c277ed6c484dabacc1a59b037e8af5" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45315 } }
I20260929 03:05:47.663854 13317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:47.664884 13323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28c277ed6c484dabacc1a59b037e8af5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c277ed6c484dabacc1a59b037e8af5" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45315 } } }
I20260929 03:05:47.665047 13323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:47.664884 13324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28c277ed6c484dabacc1a59b037e8af5. Latest consensus state: current_term: 1 leader_uuid: "28c277ed6c484dabacc1a59b037e8af5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28c277ed6c484dabacc1a59b037e8af5" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45315 } } }
I20260929 03:05:47.665280 13324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:47.665727 13331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:47.668780 13331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:47.675092 13331 catalog_manager.cc:1347] Generated new cluster ID: 826bb15a3441495e84277f2e279eb587
I20260929 03:05:47.675148 13331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:47.687016 13331 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:47.688041 13331 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:47.694818 13331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28c277ed6c484dabacc1a59b037e8af5: Generated new TSK 0
I20260929 03:05:47.695564 13331 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 03:05:47.721815 13347 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 03:05:47.721920 13347 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 03:05:47.722945 13320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:47.706472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46116 (local address 127.7.209.126:45315)
0929 03:05:47.706597 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:47.706602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:47.706810 (+ 208us) server_negotiation.cc:407] Connection header received
0929 03:05:47.706961 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:47.706964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:47.707022 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:47.707138 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0929 03:05:47.707333 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.708251 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.709121 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.709464 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.709564 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:47.709837 (+ 273us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 03:05:47.722352 (+ 12515us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 03:05:47.722767 (+ 415us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46116: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":49}
W20260929 03:05:47.722968 13346 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:05:47.706434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:45315 (local address 127.0.0.1:46116)
0929 03:05:47.706726 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:47.706739 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:47.706844 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:47.707126 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:47.707128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:47.707135 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0929 03:05:47.707247 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:47.707252 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.708354 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.708357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:47.709011 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:47.709019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.709559 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.709562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:47.709654 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:47.709679 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 03:05:47.722604 (+ 12925us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 03:05:47.722797 (+ 193us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.209.126:45315: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":121,"threads_started":1}
W20260929 03:05:47.723028 13320 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46116: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37195/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260929 03:05:47.723752 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 13250
2026-09-29T03:05:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (342 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 03:05:47.764008 8005 mini_oidc.cc:152] Starting JWKS server
I20260929 03:05:47.765007 8005 webserver.cc:460] Webserver started at https://127.0.0.1:37433/ using document root <none> and password file <none>
I20260929 03:05:47.765118 8005 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 03:05:47.765388 8005 webserver.cc:460] Webserver started at http://127.0.0.1:35755/ using document root <none> and password file <none>
2026-09-29T03:05:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:47Z Disabled control of system clock
I20260929 03:05:47.775808 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:39195
--webserver_interface=127.7.209.126
--webserver_port=0
--builtin_ntp_servers=127.7.209.84:40569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:39195
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37433/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 03:05:47.900400 13353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:47.900758 13353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:47.900818 13353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:47.904927 13353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:47.904999 13353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:47.905020 13353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:47.905041 13353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:47.905061 13353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 03:05:47.905088 13353 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 03:05:47.905114 13353 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37433/jwks.json
I20260929 03:05:47.910027 13353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:40569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:39195
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:39195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37433/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:47.911329 13353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:47.913074 13353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:47.922605 13359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:47.922689 13353 server_base.cc:1034] running on GCE node
W20260929 03:05:47.922600 13358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:47.922605 13361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:47.923483 13353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:47.924865 13353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:47.926019 13353 hybrid_clock.cc:648] HybridClock initialized: now 1790651147925998 us; error 38 us; skew 500 ppm
I20260929 03:05:47.928222 13353 webserver.cc:460] Webserver started at http://127.7.209.126:41415/ using document root <none> and password file <none>
I20260929 03:05:47.928890 13353 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:47.928960 13353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:47.929200 13353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:47.931571 13353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "7ae92dd9a6c746c8ad2e1db3f0d2bd19"
format_stamp: "Formatted at 2026-09-29 03:05:47 on dist-test-slave-bdps"
server_key: "6a07a98ae34ca7940e6a6ce133fed2b1"
server_key_iv: "8a4dc77ac520fd4fc7c1bf2782e1f8b9"
server_key_version: "encryptionkey@0"
I20260929 03:05:47.932179 13353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "7ae92dd9a6c746c8ad2e1db3f0d2bd19"
format_stamp: "Formatted at 2026-09-29 03:05:47 on dist-test-slave-bdps"
server_key: "6a07a98ae34ca7940e6a6ce133fed2b1"
server_key_iv: "8a4dc77ac520fd4fc7c1bf2782e1f8b9"
server_key_version: "encryptionkey@0"
I20260929 03:05:47.936329 13353 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260929 03:05:47.938939 13367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:47.940217 13353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 03:05:47.940330 13353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "7ae92dd9a6c746c8ad2e1db3f0d2bd19"
format_stamp: "Formatted at 2026-09-29 03:05:47 on dist-test-slave-bdps"
server_key: "6a07a98ae34ca7940e6a6ce133fed2b1"
server_key_iv: "8a4dc77ac520fd4fc7c1bf2782e1f8b9"
server_key_version: "encryptionkey@0"
I20260929 03:05:47.940423 13353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:47.953965 13353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:47.954604 13353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:47.954773 13353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:47.962880 13353 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:39195
I20260929 03:05:47.962886 13419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:39195 every 8 connection(s)
I20260929 03:05:47.964120 13353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:47.967777 13420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:47.971618 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 13353
I20260929 03:05:47.971755 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:47.972090 8005 external_mini_cluster.cc:1428] Setting key 402d83a0c9668dbe244046cb19d4f89b
I20260929 03:05:47.974884 13420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19: Bootstrap starting.
I20260929 03:05:47.977392 13420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:47.978226 13420 log.cc:824] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:47.980391 13420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19: No bootstrap required, opened a new log
I20260929 03:05:47.982499 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:47.973786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51622 (local address 127.7.209.126:39195)
0929 03:05:47.974508 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:47.974525 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:47.974567 (+ 42us) server_negotiation.cc:407] Connection header received
0929 03:05:47.975282 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:47.975305 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:47.975772 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:47.976114 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:47.976599 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.977713 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.978679 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:47.979302 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:47.979464 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:47.980256 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:47.980262 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:47.980278 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:47.980823 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:47.980895 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:47.980921 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:47.981018 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:47.981173 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 03:05:47.981452 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":130,"threads_started":1}
I20260929 03:05:47.984761 13420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae92dd9a6c746c8ad2e1db3f0d2bd19" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39195 } }
I20260929 03:05:47.985011 13420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:47.985073 13420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae92dd9a6c746c8ad2e1db3f0d2bd19, State: Initialized, Role: FOLLOWER
I20260929 03:05:47.985570 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:05:47.985517 13420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae92dd9a6c746c8ad2e1db3f0d2bd19" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39195 } }
I20260929 03:05:47.985688 13420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:47.985754 13420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:47.985852 13420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:47.986757 13420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae92dd9a6c746c8ad2e1db3f0d2bd19" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39195 } }
I20260929 03:05:47.987092 13420 leader_election.cc:302] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ae92dd9a6c746c8ad2e1db3f0d2bd19; no voters:
I20260929 03:05:47.987390 13420 leader_election.cc:288] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:47.987457 13425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:47.987699 13425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [term 1 LEADER]: Becoming Leader. State: Replica: 7ae92dd9a6c746c8ad2e1db3f0d2bd19, State: Running, Role: LEADER
I20260929 03:05:47.988013 13425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae92dd9a6c746c8ad2e1db3f0d2bd19" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39195 } }
I20260929 03:05:47.988799 13420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:47.989692 13427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ae92dd9a6c746c8ad2e1db3f0d2bd19. Latest consensus state: current_term: 1 leader_uuid: "7ae92dd9a6c746c8ad2e1db3f0d2bd19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae92dd9a6c746c8ad2e1db3f0d2bd19" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39195 } } }
I20260929 03:05:47.989817 13427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:47.990065 13426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ae92dd9a6c746c8ad2e1db3f0d2bd19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ae92dd9a6c746c8ad2e1db3f0d2bd19" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39195 } } }
I20260929 03:05:47.990170 13426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:47.990283 13434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:47.993582 13434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:47.999596 13434 catalog_manager.cc:1347] Generated new cluster ID: 0dbbff725365499ebc9bcf88382a40c5
I20260929 03:05:47.999684 13434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:48.038967 13434 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:48.040582 13434 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:48.046439 13434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ae92dd9a6c746c8ad2e1db3f0d2bd19: Generated new TSK 0
I20260929 03:05:48.047490 13434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:05:48.064389 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:48.058748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51630 (local address 127.7.209.126:39195)
0929 03:05:48.058885 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:48.058890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:48.059059 (+ 169us) server_negotiation.cc:407] Connection header received
0929 03:05:48.059221 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:48.059225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:48.059290 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:48.059422 (+ 132us) server_negotiation.cc:226] Negotiated authn=JWT
0929 03:05:48.059604 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:48.060463 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:48.061353 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:48.061758 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:48.061854 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:48.062152 (+ 298us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 03:05:48.063951 (+ 1799us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37433/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 03:05:48.064217 (+ 266us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51630: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37433/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":46}
W20260929 03:05:48.064478 13423 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51630: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37433/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260929 03:05:48.065909 13449 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:05:48.058659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39195 (local address 127.0.0.1:51630)
0929 03:05:48.058975 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:48.058989 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:48.059099 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:48.059403 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:48.059404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:48.059410 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0929 03:05:48.059521 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:48.059526 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:48.060581 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:48.060584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:48.061254 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:48.061262 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:48.061877 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:48.061880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:48.061950 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:48.061974 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 03:05:48.064141 (+ 2167us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37433/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 03:05:48.065713 (+ 1572us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.209.126:39195: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37433/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":1327,"thread_start_us":133,"threads_started":1}
I20260929 03:05:48.066334 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 13353
E20260929 03:05:48.075222 13450 webserver.cc:533] Webserver: sslize failed:
E20260929 03:05:48.075313 13450 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-29T03:05:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (342 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T03:05:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:48Z Disabled control of system clock
I20260929 03:05:48.116885 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:37883
--webserver_interface=127.7.209.126
--webserver_port=0
--builtin_ntp_servers=127.7.209.84:38019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:37883
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/testchainca.cert with env {}
W20260929 03:05:48.243901 13454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:48.244243 13454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:48.244287 13454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:48.248440 13454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:48.248498 13454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:48.248512 13454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:48.248528 13454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:48.248543 13454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 03:05:48.248556 13454 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 03:05:48.248569 13454 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 03:05:48.254068 13454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:37883
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:37883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:48.255345 13454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:48.256610 13454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:48.266686 13459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:48.266713 13454 server_base.cc:1034] running on GCE node
W20260929 03:05:48.266726 13460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:48.266752 13462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:48.267540 13454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:48.268770 13454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:48.269914 13454 hybrid_clock.cc:648] HybridClock initialized: now 1790651148269890 us; error 39 us; skew 500 ppm
I20260929 03:05:48.271844 13454 webserver.cc:460] Webserver started at http://127.7.209.126:45629/ using document root <none> and password file <none>
I20260929 03:05:48.272473 13454 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:48.272557 13454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:48.272786 13454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:48.275074 13454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "2cdb9ffbfa7c4f0280b834c015101768"
format_stamp: "Formatted at 2026-09-29 03:05:48 on dist-test-slave-bdps"
server_key: "3c711f38cde1cd81e1964db266b3847b"
server_key_iv: "9db0b664b1c629b675900d78951f05d4"
server_key_version: "encryptionkey@0"
I20260929 03:05:48.275653 13454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "2cdb9ffbfa7c4f0280b834c015101768"
format_stamp: "Formatted at 2026-09-29 03:05:48 on dist-test-slave-bdps"
server_key: "3c711f38cde1cd81e1964db266b3847b"
server_key_iv: "9db0b664b1c629b675900d78951f05d4"
server_key_version: "encryptionkey@0"
I20260929 03:05:48.279846 13454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:05:48.282585 13468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:48.283987 13454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:48.284101 13454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "2cdb9ffbfa7c4f0280b834c015101768"
format_stamp: "Formatted at 2026-09-29 03:05:48 on dist-test-slave-bdps"
server_key: "3c711f38cde1cd81e1964db266b3847b"
server_key_iv: "9db0b664b1c629b675900d78951f05d4"
server_key_version: "encryptionkey@0"
I20260929 03:05:48.284216 13454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:48.295903 13454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:48.296577 13454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:48.296744 13454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:48.304059 13454 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:37883
I20260929 03:05:48.304066 13520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:37883 every 8 connection(s)
I20260929 03:05:48.305022 13454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:48.308223 13521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:48.313391 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 13454
I20260929 03:05:48.313525 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:48.313899 8005 external_mini_cluster.cc:1428] Setting key 165b3512e7cbe7abcbbc67984c99ae51
I20260929 03:05:48.314497 13521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768: Bootstrap starting.
I20260929 03:05:48.317319 13521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:48.318329 13521 log.cc:824] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:48.320638 13521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768: No bootstrap required, opened a new log
I20260929 03:05:48.325233 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:48.315435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43348 (local address 127.7.209.126:37883)
0929 03:05:48.316167 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:48.316187 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:48.316235 (+ 48us) server_negotiation.cc:407] Connection header received
0929 03:05:48.317102 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:48.317128 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:48.317758 (+ 630us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:48.318172 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:48.318679 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:48.320104 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:48.321105 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:48.321851 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:48.322003 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:48.322864 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:48.322870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:48.322889 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:05:48.323457 (+ 568us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:48.323543 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:05:48.323571 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:48.323680 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:48.323759 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 03:05:48.324076 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":125,"threads_started":1}
I20260929 03:05:48.325521 13521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cdb9ffbfa7c4f0280b834c015101768" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37883 } }
I20260929 03:05:48.325816 13521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:48.325875 13521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cdb9ffbfa7c4f0280b834c015101768, State: Initialized, Role: FOLLOWER
I20260929 03:05:48.326373 13521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cdb9ffbfa7c4f0280b834c015101768" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37883 } }
I20260929 03:05:48.326511 13521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:48.326577 13521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:48.326678 13521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:48.327759 13521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cdb9ffbfa7c4f0280b834c015101768" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37883 } }
I20260929 03:05:48.328118 13521 leader_election.cc:302] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cdb9ffbfa7c4f0280b834c015101768; no voters:
I20260929 03:05:48.328337 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:05:48.328461 13521 leader_election.cc:288] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:48.328554 13526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:48.328768 13526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [term 1 LEADER]: Becoming Leader. State: Replica: 2cdb9ffbfa7c4f0280b834c015101768, State: Running, Role: LEADER
I20260929 03:05:48.329106 13526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cdb9ffbfa7c4f0280b834c015101768" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37883 } }
I20260929 03:05:48.329905 13521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:48.330984 13527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cdb9ffbfa7c4f0280b834c015101768" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cdb9ffbfa7c4f0280b834c015101768" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37883 } } }
I20260929 03:05:48.331131 13527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:48.331461 13528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cdb9ffbfa7c4f0280b834c015101768. Latest consensus state: current_term: 1 leader_uuid: "2cdb9ffbfa7c4f0280b834c015101768" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cdb9ffbfa7c4f0280b834c015101768" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37883 } } }
I20260929 03:05:48.331548 13528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:48.331681 13535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:48.335265 13535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:48.341163 13535 catalog_manager.cc:1347] Generated new cluster ID: 8ed633aa97e845c28c3a00758549e78c
I20260929 03:05:48.341234 13535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:48.368124 13535 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:48.369323 13535 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:48.377436 13535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cdb9ffbfa7c4f0280b834c015101768: Generated new TSK 0
I20260929 03:05:48.378332 13535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:05:48.456759 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:48.385840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43364 (local address 127.7.209.126:37883)
0929 03:05:48.385975 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:48.385980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:48.386194 (+ 214us) server_negotiation.cc:407] Connection header received
0929 03:05:48.386380 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:48.386382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:48.386433 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:48.386529 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0929 03:05:48.386758 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:48.387314 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:48.388636 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:48.388867 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:48.388946 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:48.389277 (+ 331us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 03:05:48.456253 (+ 66976us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 03:05:48.456597 (+ 344us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43364: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":46}
W20260929 03:05:48.456862 13524 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43364: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260929 03:05:48.457151 13550 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:05:48.385798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:37883 (local address 127.0.0.1:43364)
0929 03:05:48.386098 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:48.386114 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:48.386250 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:48.386537 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:48.386540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:48.386548 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0929 03:05:48.386679 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:48.386685 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:48.387413 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:48.387418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:48.388526 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:48.388538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:48.388974 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:48.388978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:48.389077 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:48.389106 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 03:05:48.456597 (+ 67491us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 03:05:48.456909 (+ 312us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.209.126:37883: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":105,"threads_started":1}
I20260929 03:05:48.457983 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 13454
2026-09-29T03:05:48Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (371 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (137 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (142 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:05:48 dist-test-slave-bdps krb5kdc[13562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:05:48 dist-test-slave-bdps krb5kdc[13562](info): set up 2 sockets
Sep 29 03:05:48 dist-test-slave-bdps krb5kdc[13562](info): commencing operation
krb5kdc: starting...
W20260929 03:05:49.690310 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:05:49 dist-test-slave-bdps krb5kdc[13562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:05:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:05:49.813864 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:34379
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:34379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:49.940008 13575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:49.940305 13575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:49.940346 13575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:49.944316 13575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:49.944367 13575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:49.944384 13575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:49.944398 13575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:49.944417 13575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:49.949535 13575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:34379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:34379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:49.950661 13575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:49.951786 13575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:49.961045 13580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:49.961182 13575 server_base.cc:1034] running on GCE node
W20260929 03:05:49.961191 13583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:49.961047 13581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:49.962078 13575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:49.963546 13575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:49.964707 13575 hybrid_clock.cc:648] HybridClock initialized: now 1790651149964682 us; error 47 us; skew 500 ppm
Sep 29 03:05:49 dist-test-slave-bdps krb5kdc[13562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651149, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:49.971364 13575 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:05:49.972431 13575 webserver.cc:460] Webserver started at http://127.7.209.126:45057/ using document root <none> and password file <none>
I20260929 03:05:49.973086 13575 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:49.973166 13575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:49.973409 13575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:49.975880 13575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "90bf7889de9b40c58f73618db04cd463"
format_stamp: "Formatted at 2026-09-29 03:05:49 on dist-test-slave-bdps"
server_key: "19c68a3ee3fedad37dc56d988690f725"
server_key_iv: "34d8706c3988d33827e23b352d05f4d1"
server_key_version: "encryptionkey@0"
I20260929 03:05:49.976497 13575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "90bf7889de9b40c58f73618db04cd463"
format_stamp: "Formatted at 2026-09-29 03:05:49 on dist-test-slave-bdps"
server_key: "19c68a3ee3fedad37dc56d988690f725"
server_key_iv: "34d8706c3988d33827e23b352d05f4d1"
server_key_version: "encryptionkey@0"
I20260929 03:05:49.980858 13575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 03:05:49.983603 13590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:49.984977 13575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:05:49.985100 13575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "90bf7889de9b40c58f73618db04cd463"
format_stamp: "Formatted at 2026-09-29 03:05:49 on dist-test-slave-bdps"
server_key: "19c68a3ee3fedad37dc56d988690f725"
server_key_iv: "34d8706c3988d33827e23b352d05f4d1"
server_key_version: "encryptionkey@0"
I20260929 03:05:49.985206 13575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:50.032720 13575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:50.036424 13575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:50.036648 13575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:50.044924 13575 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:34379
I20260929 03:05:50.044950 13642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:34379 every 8 connection(s)
I20260929 03:05:50.046144 13575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:50.049692 13643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:50.050890 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 13575
I20260929 03:05:50.051015 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:50.051343 8005 external_mini_cluster.cc:1428] Setting key 33eca014c9d4f0f957ef47b2acbadd0f
I20260929 03:05:50.057065 13643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463: Bootstrap starting.
I20260929 03:05:50.060429 13643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463: Neither blocks nor log segments found. Creating new log.
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:50.061553 13643 log.cc:824] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:50.064210 13643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463: No bootstrap required, opened a new log
I20260929 03:05:50.067314 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:50.052892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34320 (local address 127.7.209.126:34379)
0929 03:05:50.053355 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:50.053373 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:50.053418 (+ 45us) server_negotiation.cc:407] Connection header received
0929 03:05:50.054162 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:50.054183 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:50.054585 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:50.054929 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:50.055943 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.057044 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.058311 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.058907 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.059039 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:50.062269 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:50.062277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:50.062298 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:50.062331 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:50.064184 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.064722 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.064728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.064733 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.064852 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.065141 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.065143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.065144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.065582 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:50.065793 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:50.066058 (+ 265us) server_negotiation.cc:299] Negotiation successful
0929 03:05:50.066382 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":141,"threads_started":1}
I20260929 03:05:50.068766 13643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90bf7889de9b40c58f73618db04cd463" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34379 } }
I20260929 03:05:50.069025 13643 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:50.069087 13643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90bf7889de9b40c58f73618db04cd463, State: Initialized, Role: FOLLOWER
I20260929 03:05:50.069574 13643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90bf7889de9b40c58f73618db04cd463" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34379 } }
I20260929 03:05:50.069754 13643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:50.069826 13643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:50.069931 13643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:50.070977 13643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90bf7889de9b40c58f73618db04cd463" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34379 } }
I20260929 03:05:50.071374 13643 leader_election.cc:302] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90bf7889de9b40c58f73618db04cd463; no voters:
I20260929 03:05:50.071718 13643 leader_election.cc:288] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:50.071846 13648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:50.072057 13648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [term 1 LEADER]: Becoming Leader. State: Replica: 90bf7889de9b40c58f73618db04cd463, State: Running, Role: LEADER
I20260929 03:05:50.072419 13648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90bf7889de9b40c58f73618db04cd463" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34379 } }
I20260929 03:05:50.073040 13643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:50.074244 13650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90bf7889de9b40c58f73618db04cd463. Latest consensus state: current_term: 1 leader_uuid: "90bf7889de9b40c58f73618db04cd463" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90bf7889de9b40c58f73618db04cd463" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34379 } } }
I20260929 03:05:50.074368 13650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:50.074740 13649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90bf7889de9b40c58f73618db04cd463" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90bf7889de9b40c58f73618db04cd463" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34379 } } }
I20260929 03:05:50.074829 13649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:50.075632 13657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:50.078706 13657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:50.084728 13657 catalog_manager.cc:1347] Generated new cluster ID: 4524ecfe667a4e3781c9f4d699d7c73d
I20260929 03:05:50.084811 13657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:50.110505 13657 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:50.111802 13657 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:50.116830 13657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90bf7889de9b40c58f73618db04cd463: Generated new TSK 0
I20260929 03:05:50.117565 13657 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:05:50.145426 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34379
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:50.277755 13669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:50.278097 13669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:50.278170 13669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:50.282351 13669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:50.282449 13669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:50.282536 13669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:50.288283 13669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:50.289808 13669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:50.291239 13669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:50.302351 13675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:50.302387 13669 server_base.cc:1034] running on GCE node
W20260929 03:05:50.302335 13674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:50.302336 13677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:50.302991 13669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:50.303743 13669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:50.304912 13669 hybrid_clock.cc:648] HybridClock initialized: now 1790651150304885 us; error 41 us; skew 500 ppm
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:50.307958 13669 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:05:50.308940 13669 webserver.cc:460] Webserver started at http://127.7.209.65:45079/ using document root <none> and password file <none>
I20260929 03:05:50.309563 13669 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:50.309660 13669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:50.309885 13669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:50.312227 13669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "fa3928ddfd5440beb878ffa23a690a5b"
format_stamp: "Formatted at 2026-09-29 03:05:50 on dist-test-slave-bdps"
server_key: "432397371ba045309b2cf0e9885fd78c"
server_key_iv: "4e979c1cbee90031cbe57385b0baf373"
server_key_version: "encryptionkey@0"
I20260929 03:05:50.312814 13669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "fa3928ddfd5440beb878ffa23a690a5b"
format_stamp: "Formatted at 2026-09-29 03:05:50 on dist-test-slave-bdps"
server_key: "432397371ba045309b2cf0e9885fd78c"
server_key_iv: "4e979c1cbee90031cbe57385b0baf373"
server_key_version: "encryptionkey@0"
I20260929 03:05:50.317020 13669 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260929 03:05:50.319783 13684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:50.321272 13669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 03:05:50.321394 13669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "fa3928ddfd5440beb878ffa23a690a5b"
format_stamp: "Formatted at 2026-09-29 03:05:50 on dist-test-slave-bdps"
server_key: "432397371ba045309b2cf0e9885fd78c"
server_key_iv: "4e979c1cbee90031cbe57385b0baf373"
server_key_version: "encryptionkey@0"
I20260929 03:05:50.321503 13669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:50.347201 13669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:50.350390 13669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:50.350584 13669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:50.351248 13669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:50.352152 13669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:50.352209 13669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:50.352267 13669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:50.352294 13669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:50.361770 13669 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:40167
I20260929 03:05:50.361806 13797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:40167 every 8 connection(s)
I20260929 03:05:50.362893 13669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:50.364987 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 13669
I20260929 03:05:50.365135 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:05:50.365556 8005 external_mini_cluster.cc:1428] Setting key 6909bd1d318a6f1ab106dac3a275fda6
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:05:50.379742 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:50.365346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:51973 (local address 127.7.209.126:34379)
0929 03:05:50.365505 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:50.365509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:50.366816 (+ 1307us) server_negotiation.cc:407] Connection header received
0929 03:05:50.368822 (+ 2006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:50.368826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:50.368887 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:50.368966 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:50.370756 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.371620 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.372954 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.373338 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.373454 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:50.376186 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:50.376190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:50.376193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:50.376226 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:50.377815 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.378282 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.378284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.378286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.378363 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.378673 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.378676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.378678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.378890 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:50.378974 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:50.379455 (+ 481us) server_negotiation.cc:299] Negotiation successful
0929 03:05:50.379591 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
I20260929 03:05:50.380875 13800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:50.366017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:34379 (local address 127.7.209.65:51973)
0929 03:05:50.366617 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:50.366690 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:50.368498 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:50.369215 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:50.369226 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:50.369808 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:50.370558 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:50.370580 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.371750 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.371752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:50.372820 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:50.372829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.373495 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.373497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:50.373731 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:50.374413 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:50.374430 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:50.376008 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:50.377916 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:50.377921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:50.377935 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:50.378202 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:50.378430 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:50.378434 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:50.378435 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:50.378569 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:50.378969 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:50.378975 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:50.379237 (+ 262us) client_negotiation.cc:769] Sending connection context
0929 03:05:50.379410 (+ 173us) client_negotiation.cc:240] Negotiation successful
0929 03:05:50.379722 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":150,"threads_started":1}
I20260929 03:05:50.382323 13798 heartbeater.cc:347] Connected to a master server at 127.7.209.126:34379
I20260929 03:05:50.382578 13798 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:50.383104 13798 heartbeater.cc:511] Master 127.7.209.126:34379 requested a full tablet report, sending...
I20260929 03:05:50.384783 13607 ts_manager.cc:194] Registered new tserver with Master: fa3928ddfd5440beb878ffa23a690a5b (127.7.209.65:40167)
I20260929 03:05:50.386116 13607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:51973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:05:50.394614 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34379
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:50.524181 13803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:50.524588 13803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:50.524639 13803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:50.529278 13803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:50.529366 13803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:50.529464 13803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:50.535238 13803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:50.536461 13803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:50.537672 13803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:50.548130 13808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:50.548235 13803 server_base.cc:1034] running on GCE node
W20260929 03:05:50.548135 13811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:50.548164 13809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:50.548808 13803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:50.549443 13803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:50.550607 13803 hybrid_clock.cc:648] HybridClock initialized: now 1790651150550576 us; error 41 us; skew 500 ppm
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:50.553419 13803 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:05:50.554400 13803 webserver.cc:460] Webserver started at http://127.7.209.66:42211/ using document root <none> and password file <none>
I20260929 03:05:50.554993 13803 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:50.555053 13803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:50.555271 13803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:50.557518 13803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "8dbaec9eb0d74eb4a17acbe7125ceba7"
format_stamp: "Formatted at 2026-09-29 03:05:50 on dist-test-slave-bdps"
server_key: "c01d1eff7b455ff452292b9a1b205730"
server_key_iv: "2a86623fd73f27fc52fba1d5a89b8086"
server_key_version: "encryptionkey@0"
I20260929 03:05:50.558205 13803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "8dbaec9eb0d74eb4a17acbe7125ceba7"
format_stamp: "Formatted at 2026-09-29 03:05:50 on dist-test-slave-bdps"
server_key: "c01d1eff7b455ff452292b9a1b205730"
server_key_iv: "2a86623fd73f27fc52fba1d5a89b8086"
server_key_version: "encryptionkey@0"
I20260929 03:05:50.562759 13803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 03:05:50.565716 13818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:50.567330 13803 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:50.567446 13803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "8dbaec9eb0d74eb4a17acbe7125ceba7"
format_stamp: "Formatted at 2026-09-29 03:05:50 on dist-test-slave-bdps"
server_key: "c01d1eff7b455ff452292b9a1b205730"
server_key_iv: "2a86623fd73f27fc52fba1d5a89b8086"
server_key_version: "encryptionkey@0"
I20260929 03:05:50.567585 13803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:50.581271 13803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:50.584380 13803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:50.584561 13803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:50.585212 13803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:50.586177 13803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:50.586226 13803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:50.586266 13803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:50.586280 13803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:50.596019 13803 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:39639
I20260929 03:05:50.596045 13931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:39639 every 8 connection(s)
I20260929 03:05:50.597034 13803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:50.601819 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 13803
I20260929 03:05:50.601954 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:05:50.602320 8005 external_mini_cluster.cc:1428] Setting key ea3734d5516f75de780301b0310a7d1a
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:50.612835 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:50.599114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:44403 (local address 127.7.209.126:34379)
0929 03:05:50.599271 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:50.599275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:50.600280 (+ 1005us) server_negotiation.cc:407] Connection header received
0929 03:05:50.601904 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:50.601909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:50.601986 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:50.602094 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:50.603640 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.604352 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.606007 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.606242 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.606322 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:50.609099 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:50.609102 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:50.609104 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:50.609132 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:50.610702 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.611212 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.611215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.611217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.611304 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.611633 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.611637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.611638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.611846 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:50.611936 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:50.612479 (+ 543us) server_negotiation.cc:299] Negotiation successful
0929 03:05:50.612634 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:05:50.613930 13934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:50.599579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:34379 (local address 127.7.209.66:44403)
0929 03:05:50.600101 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:50.600151 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:50.601635 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:50.602287 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:50.602297 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:50.602747 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:50.603465 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:50.603482 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.604489 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.604493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:50.605893 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:50.605905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.606391 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.606395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:50.606635 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:50.607304 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:50.607322 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:50.608907 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:50.610812 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:50.610819 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:50.610837 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:50.611116 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:50.611386 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:50.611390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:50.611392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:50.611542 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:50.611944 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:50.611950 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:50.612238 (+ 288us) client_negotiation.cc:769] Sending connection context
0929 03:05:50.612418 (+ 180us) client_negotiation.cc:240] Negotiation successful
0929 03:05:50.612794 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":132,"threads_started":1}
I20260929 03:05:50.615306 13932 heartbeater.cc:347] Connected to a master server at 127.7.209.126:34379
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
I20260929 03:05:50.615644 13932 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:50.616312 13932 heartbeater.cc:511] Master 127.7.209.126:34379 requested a full tablet report, sending...
I20260929 03:05:50.617697 13607 ts_manager.cc:194] Registered new tserver with Master: 8dbaec9eb0d74eb4a17acbe7125ceba7 (127.7.209.66:39639)
I20260929 03:05:50.618574 13607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:44403
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:05:50.633175 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34379
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:41679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:50.754949 13937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:50.755244 13937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:50.755286 13937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:50.759281 13937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:50.759351 13937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:50.759407 13937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:50.765141 13937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:41679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:50.766616 13937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:50.767949 13937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:50.779675 13945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:50.779659 13942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:50.779717 13937 server_base.cc:1034] running on GCE node
W20260929 03:05:50.779695 13943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:50.780344 13937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:50.781103 13937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:50.782258 13937 hybrid_clock.cc:648] HybridClock initialized: now 1790651150782239 us; error 37 us; skew 500 ppm
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:50.785243 13937 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:05:50.786304 13937 webserver.cc:460] Webserver started at http://127.7.209.67:43339/ using document root <none> and password file <none>
I20260929 03:05:50.786952 13937 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:50.787021 13937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:50.787262 13937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:50.789731 13937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "28694213d0f04f8ebcd1c59c30ef6392"
format_stamp: "Formatted at 2026-09-29 03:05:50 on dist-test-slave-bdps"
server_key: "fd26863d19f9f93503603791a08d39a2"
server_key_iv: "124e7e5708cc0f0174c3ce60da390efc"
server_key_version: "encryptionkey@0"
I20260929 03:05:50.790401 13937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "28694213d0f04f8ebcd1c59c30ef6392"
format_stamp: "Formatted at 2026-09-29 03:05:50 on dist-test-slave-bdps"
server_key: "fd26863d19f9f93503603791a08d39a2"
server_key_iv: "124e7e5708cc0f0174c3ce60da390efc"
server_key_version: "encryptionkey@0"
I20260929 03:05:50.795029 13937 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:50.798452 13952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:50.800499 13937 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:50.800635 13937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "28694213d0f04f8ebcd1c59c30ef6392"
format_stamp: "Formatted at 2026-09-29 03:05:50 on dist-test-slave-bdps"
server_key: "fd26863d19f9f93503603791a08d39a2"
server_key_iv: "124e7e5708cc0f0174c3ce60da390efc"
server_key_version: "encryptionkey@0"
I20260929 03:05:50.800767 13937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:50.819022 13937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:50.822293 13937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:50.822469 13937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:50.823134 13937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:50.824108 13937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:50.824156 13937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:50.824196 13937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:50.824218 13937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:50.833545 13937 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:39299
I20260929 03:05:50.833599 14065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:39299 every 8 connection(s)
I20260929 03:05:50.834582 13937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:50.840513 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 13937
I20260929 03:05:50.840684 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:05:50.841049 8005 external_mini_cluster.cc:1428] Setting key d70cac1733d3d31f294a1dbb8aa71388
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:05:50.849390 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:50.836639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:37917 (local address 127.7.209.126:34379)
0929 03:05:50.836806 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:50.836810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:50.837766 (+ 956us) server_negotiation.cc:407] Connection header received
0929 03:05:50.839318 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:50.839321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:50.839382 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:50.839462 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:50.840847 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.841655 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.843111 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.843432 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.843534 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:50.846190 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:50.846194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:50.846196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:50.846227 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:50.847384 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.847881 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.847885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.847887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.847965 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.848266 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.848270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.848272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.848457 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:50.848542 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:50.849072 (+ 530us) server_negotiation.cc:299] Negotiation successful
0929 03:05:50.849209 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 03:05:50.850399 14068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:50.837060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:34379 (local address 127.7.209.67:37917)
0929 03:05:50.837560 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:50.837634 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:50.839083 (+ 1449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:50.839643 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:50.839651 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:50.840082 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:50.840677 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:50.840699 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.841786 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.841789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:50.842990 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:50.842999 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.843591 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.843593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:50.843763 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:50.844423 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:50.844441 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:50.846020 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:50.847486 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:50.847493 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:50.847512 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:50.847784 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:50.848041 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:50.848044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:50.848046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:50.848180 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:50.848548 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:50.848555 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:50.848829 (+ 274us) client_negotiation.cc:769] Sending connection context
0929 03:05:50.849017 (+ 188us) client_negotiation.cc:240] Negotiation successful
0929 03:05:50.849371 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":139,"threads_started":1}
I20260929 03:05:50.851507 14066 heartbeater.cc:347] Connected to a master server at 127.7.209.126:34379
I20260929 03:05:50.851766 14066 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:50.852291 14066 heartbeater.cc:511] Master 127.7.209.126:34379 requested a full tablet report, sending...
I20260929 03:05:50.853380 13607 ts_manager.cc:194] Registered new tserver with Master: 28694213d0f04f8ebcd1c59c30ef6392 (127.7.209.67:39299)
I20260929 03:05:50.854090 13607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:37917
I20260929 03:05:50.855132 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/test-admin.keytab.
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:50.872125 8005 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 03:05:50.875298 14073 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:05:50.873755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.7.209.65:40167)
0929 03:05:50.874110 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:50.874142 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:50.874170 (+ 28us) server_negotiation.cc:407] Connection header received
0929 03:05:50.874230 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:50.874236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:50.874454 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:50.874592 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:50.875080 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42322: BlockingRecv error: recv got EOF from 127.0.0.1:42322 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":137,"threads_started":1}
W20260929 03:05:50.875313 8005 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/krb5cc' of type FILE
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:50.876317 8005 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:50.877207 8005 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
I20260929 03:05:50.887581 14073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:50.878132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42328 (local address 127.7.209.65:40167)
0929 03:05:50.878263 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:50.878268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:50.878353 (+ 85us) server_negotiation.cc:407] Connection header received
0929 03:05:50.878481 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:50.878484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:50.878531 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:50.878638 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:50.879201 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.880077 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.880996 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.881470 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.881576 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:50.884082 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:50.884095 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:50.884114 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:50.884143 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:50.885706 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.886126 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.886132 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.886134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.886220 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.886473 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.886476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.886478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.886787 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:50.887014 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:50.887216 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 03:05:50.887402 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260929 03:05:50.891991 14073 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:05:50.891108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.7.209.65:40167)
0929 03:05:50.891249 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:50.891252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:50.891314 (+ 62us) server_negotiation.cc:407] Connection header received
0929 03:05:50.891412 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:50.891415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:50.891452 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:50.891516 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:50.891868 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42338: BlockingRecv error: recv got EOF from 127.0.0.1:42338 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260929 03:05:50.892083 8005 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/krb5cc' of type FILE
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:50.893148 8005 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:50.894035 8005 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
I20260929 03:05:50.901805 14073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:50.894855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42346 (local address 127.7.209.65:40167)
0929 03:05:50.894974 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:50.894976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:50.895017 (+ 41us) server_negotiation.cc:407] Connection header received
0929 03:05:50.895119 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:50.895121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:50.895154 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:50.895215 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:50.895610 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.896124 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.896844 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.897061 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.897129 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:50.899195 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:50.899200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:50.899203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:50.899227 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:50.900560 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.900886 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.900889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.900890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.900954 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.901123 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.901125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.901126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.901272 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:50.901369 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:50.901504 (+ 135us) server_negotiation.cc:299] Negotiation successful
0929 03:05:50.901643 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 03:05:50.904340 14073 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:05:50.903395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42350 (local address 127.7.209.65:40167)
0929 03:05:50.903499 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:50.903503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:50.903571 (+ 68us) server_negotiation.cc:407] Connection header received
0929 03:05:50.903686 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:50.903689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:50.903727 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:50.903786 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:50.904248 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42350: BlockingRecv error: recv got EOF from 127.0.0.1:42350 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260929 03:05:50.904529 8005 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790651120485886-8005-0/krb5kdc/krb5cc' of type FILE
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:50.905566 8005 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:50.906473 8005 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[13562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
I20260929 03:05:50.914167 14073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:50.907255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42354 (local address 127.7.209.65:40167)
0929 03:05:50.907363 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:50.907367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:50.907463 (+ 96us) server_negotiation.cc:407] Connection header received
0929 03:05:50.907585 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:50.907588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:50.907625 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:50.907686 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:50.908127 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.908621 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.909371 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:50.909592 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:50.909694 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:50.911619 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:50.911623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:50.911626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:50.911650 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:50.912726 (+ 1076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.913080 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.913084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.913086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.913166 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:50.913395 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:50.913398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:50.913400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:50.913552 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:50.913703 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:50.913921 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 03:05:50.914014 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 03:05:50.915114 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 13669
I20260929 03:05:50.921797 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 13803
I20260929 03:05:50.927174 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 13937
I20260929 03:05:50.932328 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 13575
2026-09-29T03:05:50Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2195 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[14086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[14086](info): set up 2 sockets
Sep 29 03:05:50 dist-test-slave-bdps krb5kdc[14086](info): commencing operation
krb5kdc: starting...
W20260929 03:05:51.890389 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:05:51 dist-test-slave-bdps krb5kdc[14086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:05:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.126.keytab.
Entry for principal oryx/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.126.keytab.
I20260929 03:05:52.022394 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:43463
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.126.keytab
--principal=oryx/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:43463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:52.148857 14099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:52.149299 14099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:52.149365 14099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:52.155145 14099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:52.155290 14099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:52.155333 14099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:52.155380 14099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:52.155424 14099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:52.163529 14099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:43463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.126.keytab
--principal=oryx/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:43463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:52.165892 14099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:52.167860 14099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:52.179558 14107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:52.179558 14104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:52.179576 14105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:52.180006 14099 server_base.cc:1034] running on GCE node
I20260929 03:05:52.180758 14099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:52.182411 14099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:52.183588 14099 hybrid_clock.cc:648] HybridClock initialized: now 1790651152183562 us; error 47 us; skew 500 ppm
Sep 29 03:05:52 dist-test-slave-bdps krb5kdc[14086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651152, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:52.187361 14099 init.cc:377] Logged in from keytab as oryx/127.7.209.126@KRBTEST.COM (short username oryx)
I20260929 03:05:52.188714 14099 webserver.cc:460] Webserver started at http://127.7.209.126:39753/ using document root <none> and password file <none>
I20260929 03:05:52.189489 14099 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:52.189560 14099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:52.189831 14099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:52.192288 14099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "073adfa114bf4d518ffcce398e022987"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "a7dd4da8eaa1dbe720b81d34d344d118"
server_key_iv: "bf6c93c8408865cd8511c015681705c0"
server_key_version: "encryptionkey@0"
I20260929 03:05:52.192987 14099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "073adfa114bf4d518ffcce398e022987"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "a7dd4da8eaa1dbe720b81d34d344d118"
server_key_iv: "bf6c93c8408865cd8511c015681705c0"
server_key_version: "encryptionkey@0"
I20260929 03:05:52.197439 14099 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 03:05:52.200260 14114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:52.201774 14099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:52.201905 14099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "073adfa114bf4d518ffcce398e022987"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "a7dd4da8eaa1dbe720b81d34d344d118"
server_key_iv: "bf6c93c8408865cd8511c015681705c0"
server_key_version: "encryptionkey@0"
I20260929 03:05:52.202023 14099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:52.215191 14099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:52.218310 14099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:52.218508 14099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:52.226296 14099 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:43463
I20260929 03:05:52.226567 14166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:43463 every 8 connection(s)
I20260929 03:05:52.227373 14099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:52.229997 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 14099
I20260929 03:05:52.230154 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:52.230526 8005 external_mini_cluster.cc:1428] Setting key 8df76782c08bf1cd0a92371ef96efb32
I20260929 03:05:52.230865 14167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:52.238461 14167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987: Bootstrap starting.
Sep 29 03:05:52 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:52.241333 14167 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:52.242170 14167 log.cc:824] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:52.244356 14167 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987: No bootstrap required, opened a new log
I20260929 03:05:52.245487 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:52.232066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40706 (local address 127.7.209.126:43463)
0929 03:05:52.232487 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:52.232504 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:52.232545 (+ 41us) server_negotiation.cc:407] Connection header received
0929 03:05:52.233271 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:52.233290 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:52.233688 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:52.234004 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:52.234902 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:52.235956 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:52.237145 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:52.237675 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:52.237791 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:52.240599 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:52.240604 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:52.240619 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:52.240647 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:52.242450 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:52.242911 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:52.242916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:52.242921 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:52.243040 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:52.243317 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:52.243320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:52.243322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:52.243771 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:52.243960 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:52.244273 (+ 313us) server_negotiation.cc:299] Negotiation successful
0929 03:05:52.244617 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":124,"threads_started":1}
I20260929 03:05:52.248757 14167 raft_consensus.cc:348] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } }
I20260929 03:05:52.248996 14167 raft_consensus.cc:374] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:52.249053 14167 raft_consensus.cc:729] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073adfa114bf4d518ffcce398e022987, State: Initialized, Role: FOLLOWER
I20260929 03:05:52.249511 14167 consensus_queue.cc:260] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } }
I20260929 03:05:52.249670 14167 raft_consensus.cc:388] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:52.249737 14167 raft_consensus.cc:482] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:52.249838 14167 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:52.250849 14167 raft_consensus.cc:504] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } }
I20260929 03:05:52.251214 14167 leader_election.cc:302] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 073adfa114bf4d518ffcce398e022987; no voters:
I20260929 03:05:52.251531 14167 leader_election.cc:288] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:52.251628 14172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:52.251838 14172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 1 LEADER]: Becoming Leader. State: Replica: 073adfa114bf4d518ffcce398e022987, State: Running, Role: LEADER
I20260929 03:05:52.252151 14172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } }
I20260929 03:05:52.252966 14167 sys_catalog.cc:564] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:52.253999 14173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } }
I20260929 03:05:52.254135 14173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:52.254539 14181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:52.254526 14174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 073adfa114bf4d518ffcce398e022987. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } }
I20260929 03:05:52.254739 14174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:52.257831 14181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:52.263370 14181 catalog_manager.cc:1347] Generated new cluster ID: 8689b40822d54e7ab5d5845ac4e6d649
I20260929 03:05:52.263434 14181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:52.282022 14181 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:52.283263 14181 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:52.290521 14181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987: Generated new TSK 0
I20260929 03:05:52.291718 14181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab.
Entry for principal oryx/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab.
I20260929 03:05:52.332597 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:43463
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab
--principal=oryx/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:52.459807 14193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:52.460145 14193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:52.460201 14193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:52.464340 14193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:52.464402 14193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:52.464468 14193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:52.469909 14193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab
--principal=oryx/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:43463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:52.471155 14193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:52.472370 14193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:52.483283 14201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:52.483283 14198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:52.483402 14193 server_base.cc:1034] running on GCE node
W20260929 03:05:52.483287 14199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:52.484091 14193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:52.484941 14193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:52.486132 14193 hybrid_clock.cc:648] HybridClock initialized: now 1790651152486113 us; error 40 us; skew 500 ppm
Sep 29 03:05:52 dist-test-slave-bdps krb5kdc[14086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651152, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:52.489521 14193 init.cc:377] Logged in from keytab as oryx/127.7.209.65@KRBTEST.COM (short username oryx)
I20260929 03:05:52.490684 14193 webserver.cc:460] Webserver started at http://127.7.209.65:46805/ using document root <none> and password file <none>
I20260929 03:05:52.491390 14193 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:52.491460 14193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:52.491720 14193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:52.494307 14193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "e244ba09d89d4662a64088da114ca347"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "eeb2fefe4aa870d6bb93074501e7a442"
server_key_iv: "51116ae655a8ea3bdc6339449aedddc1"
server_key_version: "encryptionkey@0"
I20260929 03:05:52.494907 14193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "e244ba09d89d4662a64088da114ca347"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "eeb2fefe4aa870d6bb93074501e7a442"
server_key_iv: "51116ae655a8ea3bdc6339449aedddc1"
server_key_version: "encryptionkey@0"
I20260929 03:05:52.499336 14193 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:05:52.501971 14208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:52.503549 14193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:52.503657 14193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "e244ba09d89d4662a64088da114ca347"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "eeb2fefe4aa870d6bb93074501e7a442"
server_key_iv: "51116ae655a8ea3bdc6339449aedddc1"
server_key_version: "encryptionkey@0"
I20260929 03:05:52.503767 14193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:52.529341 14193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:52.533119 14193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:52.533326 14193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:52.534049 14193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:52.535027 14193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:52.535074 14193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:52.535121 14193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:52.535151 14193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:52.544492 14193 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:39031
I20260929 03:05:52.544519 14321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:39031 every 8 connection(s)
I20260929 03:05:52.545470 14193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:52.550736 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 14193
I20260929 03:05:52.550853 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:05:52.551167 8005 external_mini_cluster.cc:1428] Setting key c498d4d460825afc91b92d6f2bcd8e68
Sep 29 03:05:52 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651152, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:52.560279 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:52.547499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:53505 (local address 127.7.209.126:43463)
0929 03:05:52.547661 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:52.547665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:52.548568 (+ 903us) server_negotiation.cc:407] Connection header received
0929 03:05:52.549947 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:52.549951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:52.550006 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:52.550083 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:52.551404 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:52.552112 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:52.553381 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:52.553754 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:52.553849 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:52.556574 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:52.556578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:52.556581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:52.556609 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:52.558108 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:52.558634 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:52.558637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:52.558639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:52.558739 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:52.559048 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:52.559051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:52.559053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:52.559315 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:52.559426 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:52.559912 (+ 486us) server_negotiation.cc:299] Negotiation successful
0929 03:05:52.560067 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 03:05:52.561156 14324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:52.547953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.65:53505)
0929 03:05:52.548411 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:52.548457 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:52.549742 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:52.550254 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:52.550262 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:52.550629 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:52.551256 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:52.551273 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:52.552235 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:52.552240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:52.553262 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:52.553273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:52.553913 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:52.553918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:52.554100 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:52.554650 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:52.554666 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:52.556419 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:52.558239 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:52.558246 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:52.558264 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:52.558541 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:52.558827 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:52.558829 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:52.558830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:52.558962 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:52.559409 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:52.559414 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:52.559672 (+ 258us) client_negotiation.cc:769] Sending connection context
0929 03:05:52.559837 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 03:05:52.560122 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260929 03:05:52.562551 14322 heartbeater.cc:347] Connected to a master server at 127.7.209.126:43463
WARNING: no policy specified for oryx/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:05:52.562849 14322 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:52.563465 14322 heartbeater.cc:511] Master 127.7.209.126:43463 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.66@KRBTEST.COM" created.
I20260929 03:05:52.565522 14131 ts_manager.cc:194] Registered new tserver with Master: e244ba09d89d4662a64088da114ca347 (127.7.209.65:39031)
I20260929 03:05:52.567283 14131 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:53505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab.
Entry for principal oryx/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab.
I20260929 03:05:52.581879 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:43463
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab
--principal=oryx/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:52.708760 14327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:52.709069 14327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:52.709136 14327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:52.713140 14327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:52.713196 14327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:52.713279 14327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:52.718479 14327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab
--principal=oryx/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:43463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:52.719743 14327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:52.721097 14327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:52.732577 14335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:52.732663 14332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:52.732654 14333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:52.732791 14327 server_base.cc:1034] running on GCE node
I20260929 03:05:52.733300 14327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:52.734100 14327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:52.735286 14327 hybrid_clock.cc:648] HybridClock initialized: now 1790651152735254 us; error 47 us; skew 500 ppm
Sep 29 03:05:52 dist-test-slave-bdps krb5kdc[14086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651152, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:52.738363 14327 init.cc:377] Logged in from keytab as oryx/127.7.209.66@KRBTEST.COM (short username oryx)
I20260929 03:05:52.739323 14327 webserver.cc:460] Webserver started at http://127.7.209.66:40047/ using document root <none> and password file <none>
I20260929 03:05:52.739945 14327 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:52.740003 14327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:52.740211 14327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:52.742578 14327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "8b38a7e3aef14c47826af2d89ef37697"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "7c9d0770ac307584c6e7e408fa953b0e"
server_key_iv: "f02dde08d823245e76f6a995848106ca"
server_key_version: "encryptionkey@0"
I20260929 03:05:52.743184 14327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "8b38a7e3aef14c47826af2d89ef37697"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "7c9d0770ac307584c6e7e408fa953b0e"
server_key_iv: "f02dde08d823245e76f6a995848106ca"
server_key_version: "encryptionkey@0"
I20260929 03:05:52.747291 14327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:05:52.749982 14342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:52.751400 14327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:05:52.751504 14327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "8b38a7e3aef14c47826af2d89ef37697"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "7c9d0770ac307584c6e7e408fa953b0e"
server_key_iv: "f02dde08d823245e76f6a995848106ca"
server_key_version: "encryptionkey@0"
I20260929 03:05:52.751593 14327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:52.767863 14327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:52.771332 14327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:52.771513 14327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:52.772188 14327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:52.773162 14327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:52.773214 14327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:52.773259 14327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:52.773279 14327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:52.783017 14327 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:34659
I20260929 03:05:52.783041 14455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:34659 every 8 connection(s)
I20260929 03:05:52.784013 14327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:52.789532 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 14327
I20260929 03:05:52.789700 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:05:52.790078 8005 external_mini_cluster.cc:1428] Setting key 56b72d5a861a5faeeccdce22d0bf1124
Sep 29 03:05:52 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651152, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:52.799866 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:52.786125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:50029 (local address 127.7.209.126:43463)
0929 03:05:52.786283 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:52.786287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:52.787270 (+ 983us) server_negotiation.cc:407] Connection header received
0929 03:05:52.788677 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:52.788680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:52.788733 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:52.788812 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:52.790453 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:52.791192 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:52.792742 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:52.792976 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:52.793062 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:52.796036 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:52.796039 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:52.796041 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:52.796070 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:52.797562 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:52.798202 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:52.798205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:52.798206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:52.798293 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:52.798653 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:52.798655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:52.798656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:52.798843 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:52.798917 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:52.799592 (+ 675us) server_negotiation.cc:299] Negotiation successful
0929 03:05:52.799708 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 03:05:52.801435 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:52.786630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.66:50029)
0929 03:05:52.787103 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:52.787150 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:52.788454 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:52.788997 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:52.789004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:52.789486 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:52.790294 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:52.790317 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:52.791315 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:52.791319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:52.792610 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:52.792621 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:52.793121 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:52.793125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:52.793322 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:52.793971 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:52.793991 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:52.795872 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:52.797709 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:52.797716 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:52.797734 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:52.798093 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:52.798369 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:52.798373 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:52.798376 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:52.798568 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:52.798936 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:52.798944 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:52.799325 (+ 381us) client_negotiation.cc:769] Sending connection context
0929 03:05:52.799567 (+ 242us) client_negotiation.cc:240] Negotiation successful
0929 03:05:52.799999 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for oryx/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:05:52.802958 14456 heartbeater.cc:347] Connected to a master server at 127.7.209.126:43463
I20260929 03:05:52.803272 14456 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:52.803949 14456 heartbeater.cc:511] Master 127.7.209.126:43463 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.67@KRBTEST.COM" created.
I20260929 03:05:52.805395 14131 ts_manager.cc:194] Registered new tserver with Master: 8b38a7e3aef14c47826af2d89ef37697 (127.7.209.66:34659)
I20260929 03:05:52.806141 14131 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:50029
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab.
Entry for principal oryx/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab.
I20260929 03:05:52.820369 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:43463
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab
--principal=oryx/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:52.945159 14461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:52.945443 14461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:52.945484 14461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:52.949379 14461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:52.949426 14461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:52.949481 14461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:52.954603 14461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab
--principal=oryx/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:43463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:52.955765 14461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:52.956980 14461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:52.968338 14467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:52.968487 14461 server_base.cc:1034] running on GCE node
W20260929 03:05:52.968338 14466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:52.968366 14469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:52.969183 14461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:52.969995 14461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:52.971163 14461 hybrid_clock.cc:648] HybridClock initialized: now 1790651152971140 us; error 34 us; skew 500 ppm
Sep 29 03:05:52 dist-test-slave-bdps krb5kdc[14086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651152, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:52.974260 14461 init.cc:377] Logged in from keytab as oryx/127.7.209.67@KRBTEST.COM (short username oryx)
I20260929 03:05:52.975286 14461 webserver.cc:460] Webserver started at http://127.7.209.67:35023/ using document root <none> and password file <none>
I20260929 03:05:52.975931 14461 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:52.976018 14461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:52.976274 14461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:52.978688 14461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "c49acfe4441c4b0a84aa8e41ac0cc054"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "8f0434620c86bc717f11f76c6b254275"
server_key_iv: "56ee47d1db72c4226688cb3738395292"
server_key_version: "encryptionkey@0"
I20260929 03:05:52.979256 14461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "c49acfe4441c4b0a84aa8e41ac0cc054"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "8f0434620c86bc717f11f76c6b254275"
server_key_iv: "56ee47d1db72c4226688cb3738395292"
server_key_version: "encryptionkey@0"
I20260929 03:05:52.983552 14461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:05:52.986173 14476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:52.987589 14461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:52.987696 14461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "c49acfe4441c4b0a84aa8e41ac0cc054"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "8f0434620c86bc717f11f76c6b254275"
server_key_iv: "56ee47d1db72c4226688cb3738395292"
server_key_version: "encryptionkey@0"
I20260929 03:05:52.987797 14461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:53.008291 14461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:53.011790 14461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:53.011993 14461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:53.012753 14461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:53.013796 14461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:53.013849 14461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:53.013906 14461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:53.013935 14461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:53.023578 14461 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:37215
I20260929 03:05:53.023607 14589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:37215 every 8 connection(s)
I20260929 03:05:53.024601 14461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:53.027679 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 14461
I20260929 03:05:53.027843 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:05:53.028255 8005 external_mini_cluster.cc:1428] Setting key a52e1e4826ac965b553bdd46410f685f
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651152, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:53.039067 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.026628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:36193 (local address 127.7.209.126:43463)
0929 03:05:53.026809 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.026814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.027884 (+ 1070us) server_negotiation.cc:407] Connection header received
0929 03:05:53.029506 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.029509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.029574 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.029723 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.031399 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.031970 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.033117 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.033341 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.033414 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.036012 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.036015 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.036018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.036043 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.037296 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.037784 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.037787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.037788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.037865 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.038130 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.038132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.038133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.038317 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.038389 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.038820 (+ 431us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.038929 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 03:05:53.039983 14592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.027111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.67:36193)
0929 03:05:53.027689 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.027754 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.029268 (+ 1514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.030077 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.030087 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.030632 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.031256 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.031273 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.032068 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.032071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.033029 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.033037 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.033449 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.033452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.033601 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.034257 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.034274 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.035857 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.037395 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.037400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.037412 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.037713 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.037929 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.037931 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.037932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.038069 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.038386 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.038390 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.038630 (+ 240us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.038791 (+ 161us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.039063 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":115,"threads_started":1}
I20260929 03:05:53.041164 14590 heartbeater.cc:347] Connected to a master server at 127.7.209.126:43463
I20260929 03:05:53.041453 14590 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:53.042129 14590 heartbeater.cc:511] Master 127.7.209.126:43463 requested a full tablet report, sending...
I20260929 03:05:53.043329 14131 ts_manager.cc:194] Registered new tserver with Master: c49acfe4441c4b0a84aa8e41ac0cc054 (127.7.209.67:37215)
I20260929 03:05:53.043985 14131 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:36193
I20260929 03:05:53.054155 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.209.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.125.keytab.
Entry for principal oryx/127.7.209.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.125.keytab.
I20260929 03:05:53.081077 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.125:44209
--webserver_interface=127.7.209.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.125.keytab
--principal=oryx/127.7.209.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:43463,127.7.209.125:44209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:53.207522 14595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:53.207897 14595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:53.207962 14595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:53.212280 14595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:53.212363 14595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:53.212390 14595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:53.212412 14595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:53.212445 14595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:53.218271 14595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:43463,127.7.209.125:44209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.125.keytab
--principal=oryx/127.7.209.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.125:44209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.209.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
W20260929 03:05:53.218544 14595 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.209.126:43463,127.7.209.125:44209'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 03:05:53.219689 14595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:53.220980 14595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:53.230309 14603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:53.230324 14601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:53.230324 14600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:53.231098 14595 server_base.cc:1034] running on GCE node
I20260929 03:05:53.231665 14595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:53.232853 14595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:53.234018 14595 hybrid_clock.cc:648] HybridClock initialized: now 1790651153233989 us; error 48 us; skew 500 ppm
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:53.237222 14595 init.cc:377] Logged in from keytab as oryx/127.7.209.125@KRBTEST.COM (short username oryx)
I20260929 03:05:53.238183 14595 webserver.cc:460] Webserver started at http://127.7.209.125:45105/ using document root <none> and password file <none>
I20260929 03:05:53.238767 14595 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:53.238832 14595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:53.239044 14595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:53.241285 14595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data/instance:
uuid: "1b84e1dc781d456c90c2ceefed504316"
format_stamp: "Formatted at 2026-09-29 03:05:53 on dist-test-slave-bdps"
server_key: "e23a1022dff8bc29b66b8c61c42fc040"
server_key_iv: "ac62023bd815f2f6d7edb67f89e9d464"
server_key_version: "encryptionkey@0"
I20260929 03:05:53.241906 14595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/wal/instance:
uuid: "1b84e1dc781d456c90c2ceefed504316"
format_stamp: "Formatted at 2026-09-29 03:05:53 on dist-test-slave-bdps"
server_key: "e23a1022dff8bc29b66b8c61c42fc040"
server_key_iv: "ac62023bd815f2f6d7edb67f89e9d464"
server_key_version: "encryptionkey@0"
I20260929 03:05:53.246172 14595 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:05:53.248854 14610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:53.250329 14595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:53.250447 14595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/wal
uuid: "1b84e1dc781d456c90c2ceefed504316"
format_stamp: "Formatted at 2026-09-29 03:05:53 on dist-test-slave-bdps"
server_key: "e23a1022dff8bc29b66b8c61c42fc040"
server_key_iv: "ac62023bd815f2f6d7edb67f89e9d464"
server_key_version: "encryptionkey@0"
I20260929 03:05:53.250547 14595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:53.262555 14595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:53.265592 14595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:53.265808 14595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:53.272996 14595 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.125:44209
I20260929 03:05:53.273285 14662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.125:44209 every 8 connection(s)
I20260929 03:05:53.274096 14595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
I20260929 03:05:53.277470 14663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:53.278685 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 14595
I20260929 03:05:53.282905 14663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.209.125@KRBTEST.COM
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.125@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:53.295442 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.285308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40712 (local address 127.7.209.126:43463)
0929 03:05:53.285488 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.285494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.285869 (+ 375us) server_negotiation.cc:407] Connection header received
0929 03:05:53.286033 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.286036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.286094 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.286171 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.286981 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.287735 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.289006 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.289240 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.289344 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.291991 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.291994 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.291996 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.292023 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.293535 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.294020 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.294024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.294026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.294110 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.294420 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.294423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.294425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.294754 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.294849 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.295166 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.295284 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 03:05:53.295790 14666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.280544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44104 (local address 127.7.209.125:44209)
0929 03:05:53.281043 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.281064 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.281130 (+ 66us) server_negotiation.cc:407] Connection header received
0929 03:05:53.282088 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.282127 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.282624 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.283066 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.283915 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.285314 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.286599 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.287214 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.287372 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.290501 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.290506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.290519 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.290540 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.292337 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.292787 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.292792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.292798 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.292936 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.293192 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.293195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.293197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.293801 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.293988 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.294291 (+ 303us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.294638 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"spinlock_wait_cycles":4352,"thread_start_us":136,"threads_started":1}
I20260929 03:05:53.295786 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.285227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:40712)
0929 03:05:53.285722 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.285782 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.285920 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.286247 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.286253 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.286720 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.286855 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.286861 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.287866 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.287869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.288897 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.288908 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.289386 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.289389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.289472 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.290093 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.290115 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.291851 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.293684 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.293691 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.293696 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.293920 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.294217 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.294221 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.294222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.294319 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.294854 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.294860 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.294958 (+ 98us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.295069 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.295220 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":154,"threads_started":1}
I20260929 03:05:53.297240 14663 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.125@KRBTEST.COM for oryx/127.7.209.125@KRBTEST.COM
I20260929 03:05:53.304442 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.297756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:44209 (local address 127.0.0.1:44112)
0929 03:05:53.297858 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.297872 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.297933 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.298331 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.298333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.298453 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.298893 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.298899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.299534 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.299536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.300033 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.300039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.300379 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.300381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.300459 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.300840 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.300850 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.301990 (+ 1140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.303402 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.303404 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.303405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.303571 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.303761 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.303763 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.303764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.303827 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.304080 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.304081 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.304144 (+ 63us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.304217 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.304310 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":8}
I20260929 03:05:53.304469 14666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.297804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44112 (local address 127.7.209.125:44209)
0929 03:05:53.297903 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.297907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.297920 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:05:53.298193 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.298196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.298241 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.298314 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.298976 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.299457 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.300104 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.300310 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.300378 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.302107 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.302109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.302111 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.302130 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.303320 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.303637 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.303639 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.303641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.303703 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.303884 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.303886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.303887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.304021 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.304087 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.304255 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.304360 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":42}
I20260929 03:05:53.307606 14663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: Bootstrap starting.
I20260929 03:05:53.310406 14663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:53.311236 14663 log.cc:824] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:53.313350 14663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: No bootstrap required, opened a new log
I20260929 03:05:53.316709 14663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } }
I20260929 03:05:53.316946 14663 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:53.317006 14663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b84e1dc781d456c90c2ceefed504316, State: Initialized, Role: FOLLOWER
I20260929 03:05:53.317500 14663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } }
I20260929 03:05:53.317991 14669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } } }
I20260929 03:05:53.318110 14669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:05:53.318861 14663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 03:05:53.324237 14680 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:05:53.324333 14680 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:05:53.326313 14595 master_runner.cc:428] Detected that this master 1b84e1dc781d456c90c2ceefed504316 is joining an existing cluster
I20260929 03:05:53.326385 14595 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.209.125:44209
I20260929 03:05:53.337323 14666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.329774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.7.209.125:44209)
0929 03:05:53.329876 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.329880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.329961 (+ 81us) server_negotiation.cc:407] Connection header received
0929 03:05:53.330063 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.330066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.330102 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.330172 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.330614 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.331366 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.332399 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.332611 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.332698 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.334372 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.334375 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.334378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.334398 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.335872 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.336236 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.336239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.336241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.336314 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.336541 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.336544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.336545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.336713 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.336796 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.337020 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.337160 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":42,"spinlock_wait_cycles":1792}
I20260929 03:05:53.337335 14687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.329494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:44209 (local address 127.0.0.1:44114)
0929 03:05:53.329876 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.329888 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.329991 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.330183 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.330186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.330397 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.330495 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.330500 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.331481 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.331484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.332315 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.332322 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.332710 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.332713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.332794 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.333870 (+ 1076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.333884 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.334273 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.335964 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.335967 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.335969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.336156 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.336389 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.336392 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.336394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.336469 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.336793 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.336796 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.336886 (+ 90us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.336968 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.337110 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"spinlock_wait_cycles":768,"thread_start_us":124,"threads_started":1}
I20260929 03:05:53.337927 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.329136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40722 (local address 127.7.209.126:43463)
0929 03:05:53.329296 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.329302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.329410 (+ 108us) server_negotiation.cc:407] Connection header received
0929 03:05:53.329653 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.329657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.329763 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.329827 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.330573 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.331166 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.332314 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.332700 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.332810 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.334277 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.334282 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.334285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.334313 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.336157 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.336621 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.336625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.336627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.336725 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.337005 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.337008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.337010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.337220 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.337287 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.337630 (+ 343us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.337764 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 03:05:53.338377 14686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.329011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:40722)
0929 03:05:53.329316 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.329332 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.329490 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.330003 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.330006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.330300 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.330448 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.330456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.331267 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.331270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.332190 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.332198 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.332837 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.332840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.332923 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.333562 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.333578 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.334114 (+ 536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.336277 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.336280 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.336283 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.336515 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.336829 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.336831 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.336834 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.336913 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.337354 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.337357 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.337443 (+ 86us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.338074 (+ 631us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.338208 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":113,"threads_started":1}
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651152, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.126@KRBTEST.COM for oryx/127.7.209.125@KRBTEST.COM
I20260929 03:05:53.352162 14666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.342911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44128 (local address 127.7.209.125:44209)
0929 03:05:53.343011 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.343015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.343462 (+ 447us) server_negotiation.cc:407] Connection header received
0929 03:05:53.343658 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.343660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.343701 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.343756 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.344501 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.344971 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.346216 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.346413 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.346480 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.348863 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.348867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.348870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.348893 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.350435 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.350928 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.350931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.350933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.351009 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.351260 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.351263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.351264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.351452 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.351540 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.351911 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.352014 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 03:05:53.352224 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.342883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:44209 (local address 127.0.0.1:44128)
0929 03:05:53.343311 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.343378 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.343521 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.343804 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.343810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.344271 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.344409 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.344416 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.345059 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.345062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.346111 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.346122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.346504 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.346508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.346621 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.347083 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.347097 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.348743 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.350541 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.350552 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.350560 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.350841 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.351094 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.351097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.351099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.351184 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.351546 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.351552 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.351693 (+ 141us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.351858 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.352059 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":107,"threads_started":1}
I20260929 03:05:53.353870 14131 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.209.125:44209: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1b84e1dc781d456c90c2ceefed504316"
member_type: NON_VOTER
last_known_addr {
host: "127.7.209.125"
port: 44209
}
attrs {
promote: true
}
}
dest_uuid: "073adfa114bf4d518ffcce398e022987"
cas_config_opid_index: -1
I20260929 03:05:53.354725 14131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true } }
I20260929 03:05:53.357013 14692 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 1b84e1dc781d456c90c2ceefed504316 (127.7.209.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true } } }
I20260929 03:05:53.357697 14637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:53.359263 14637 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 FOLLOWER]: Refusing update from remote peer 073adfa114bf4d518ffcce398e022987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 03:05:53.359531 14692 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.209.125:44209
WARNING: no policy specified for oryx/127.7.209.124@KRBTEST.COM; defaulting to no policy
I20260929 03:05:53.360066 14595 master.cc:537] Master@127.7.209.125:44209 shutting down...
I20260929 03:05:53.360317 14694 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 03:05:53.362675 14117 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.124@KRBTEST.COM" created.
I20260929 03:05:53.363021 14691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true } } }
I20260929 03:05:53.363148 14691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:53.366135 14595 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260929 03:05:53.366235 14117 consensus_peers.cc:487] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 -> Peer 1b84e1dc781d456c90c2ceefed504316 (127.7.209.125:44209): Couldn't send request to peer 1b84e1dc781d456c90c2ceefed504316. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 03:05:53.366490 14595 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:53.366613 14595 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: stopping tablet replica
I20260929 03:05:53.371191 14595 master.cc:552] Master@127.7.209.125:44209 shutdown complete.
I20260929 03:05:53.371241 14595 master_runner.cc:305] Clearing existing system tablet
I20260929 03:05:53.371480 14595 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:53.371531 14595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:53.372640 14595 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 03:05:53.373427 14699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:53.373643 14595 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:53.373767 14595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:53.373804 14595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/wal
uuid: "1b84e1dc781d456c90c2ceefed504316"
format_stamp: "Formatted at 2026-09-29 03:05:53 on dist-test-slave-bdps"
server_key: "e23a1022dff8bc29b66b8c61c42fc040"
server_key_iv: "ac62023bd815f2f6d7edb67f89e9d464"
server_key_version: "encryptionkey@0"
I20260929 03:05:53.375967 14595 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:53.379552 14595 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 03:05:53.379643 14595 log.cc:1196] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 03:05:53.379884 14595 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: Deleting consensus metadata
I20260929 03:05:53.380059 14595 master_runner.cc:315] Copying system tablet from 127.7.209.126:43463
I20260929 03:05:53.381397 14595 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: tablet copy: Beginning tablet copy session from remote peer at address 127.7.209.126:43463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.124.keytab.
Entry for principal oryx/127.7.209.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.124.keytab.
I20260929 03:05:53.384209 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.124:35661
--webserver_interface=127.7.209.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.124.keytab
--principal=oryx/127.7.209.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:43463,127.7.209.125:44209,127.7.209.124:35661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:05:53.390034 14705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.382076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:40730)
0929 03:05:53.382506 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.382523 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.382689 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.382957 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.382960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.383204 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.383360 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.383368 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.384226 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.384230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.384950 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.384960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.385425 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.385429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.385523 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.386218 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.386236 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.386695 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.388521 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.388524 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.388527 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.388733 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.388993 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.388996 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.388998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.389077 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.389441 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.389444 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.389543 (+ 99us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.389660 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.389820 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":108,"threads_started":1}
I20260929 03:05:53.390025 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.382202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40730 (local address 127.7.209.126:43463)
0929 03:05:53.382380 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.382385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.382614 (+ 229us) server_negotiation.cc:407] Connection header received
0929 03:05:53.382793 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.382795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.382850 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.382925 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.383496 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.384104 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.385071 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.385315 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.385409 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.386850 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.386854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.386857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.386890 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.388417 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.388822 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.388826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.388828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.388913 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.389155 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.389158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.389160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.389354 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.389441 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.389728 (+ 287us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.389847 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 03:05:53.390925 14151 tablet_copy_service.cc:140] P 073adfa114bf4d518ffcce398e022987: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1b84e1dc781d456c90c2ceefed504316 ({username='oryx', principal='oryx/127.7.209.125@KRBTEST.COM'} at 127.0.0.1:40730)
I20260929 03:05:53.391115 14151 tablet_copy_service.cc:161] P 073adfa114bf4d518ffcce398e022987: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1b84e1dc781d456c90c2ceefed504316 at {username='oryx', principal='oryx/127.7.209.125@KRBTEST.COM'} at 127.0.0.1:40730: session id = 1b84e1dc781d456c90c2ceefed504316-00000000000000000000000000000000
I20260929 03:05:53.392974 14151 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 03:05:53.394716 14595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 03:05:53.396814 14595 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: tablet copy: Starting download of 0 data blocks...
I20260929 03:05:53.397001 14595 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: tablet copy: Starting download of 1 WAL segments...
I20260929 03:05:53.398653 14595 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 03:05:53.400270 14151 tablet_copy_service.cc:342] P 073adfa114bf4d518ffcce398e022987: Request end of tablet copy session 1b84e1dc781d456c90c2ceefed504316-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.209.125@KRBTEST.COM'} at 127.0.0.1:40730
I20260929 03:05:53.400372 14151 tablet_copy_service.cc:433] P 073adfa114bf4d518ffcce398e022987: ending tablet copy session 1b84e1dc781d456c90c2ceefed504316-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1b84e1dc781d456c90c2ceefed504316
I20260929 03:05:53.408269 14595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:53.409979 14710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:53.410056 14707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:53.410476 14708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:53.410509 14595 server_base.cc:1034] running on GCE node
I20260929 03:05:53.410792 14595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:53.411083 14595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:53.412215 14595 hybrid_clock.cc:648] HybridClock initialized: now 1790651153412213 us; error 33 us; skew 500 ppm
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:53.413743 14595 init.cc:377] Logged in from keytab as oryx/127.7.209.125@KRBTEST.COM (short username oryx)
I20260929 03:05:53.414199 14595 webserver.cc:460] Webserver started at http://127.7.209.125:37677/ using document root <none> and password file <none>
I20260929 03:05:53.414383 14595 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:53.414417 14595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:53.415520 14595 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260929 03:05:53.416404 14717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:53.416616 14595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260929 03:05:53.416692 14595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/wal
uuid: "1b84e1dc781d456c90c2ceefed504316"
format_stamp: "Formatted at 2026-09-29 03:05:53 on dist-test-slave-bdps"
server_key: "e23a1022dff8bc29b66b8c61c42fc040"
server_key_iv: "ac62023bd815f2f6d7edb67f89e9d464"
server_key_version: "encryptionkey@0"
I20260929 03:05:53.416783 14595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:53.423336 14595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:53.425020 14595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:53.429819 14595 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.125:44209
I20260929 03:05:53.430408 14595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
I20260929 03:05:53.430878 14769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.125:44209 every 8 connection(s)
I20260929 03:05:53.432085 14770 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:05:53.432904 14770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: Bootstrap starting.
I20260929 03:05:53.441735 14770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:05:53.442029 14770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: Bootstrap complete.
I20260929 03:05:53.442571 14770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true } }
I20260929 03:05:53.442700 14770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1b84e1dc781d456c90c2ceefed504316, State: Initialized, Role: LEARNER
I20260929 03:05:53.442808 14770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true } }
I20260929 03:05:53.443149 14773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true } } }
I20260929 03:05:53.443311 14770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:53.443332 14773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: This master's current role is: LEARNER
I20260929 03:05:53.446208 14784 catalog_manager.cc:1259] Loaded cluster ID: 8689b40822d54e7ab5d5845ac4e6d649
I20260929 03:05:53.446331 14784 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: loading cluster ID for follower catalog manager: success
I20260929 03:05:53.448330 14784 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: acquiring CA information for follower catalog manager: success
I20260929 03:05:53.449509 14784 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 03:05:53.557351 14706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:53.557675 14706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:53.557723 14706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:53.561709 14706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:53.561790 14706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:53.561810 14706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:53.561825 14706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:53.561839 14706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:53.567792 14706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:43463,127.7.209.125:44209,127.7.209.124:35661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.124.keytab
--principal=oryx/127.7.209.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.124:35661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.209.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:53.569221 14706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:53.570359 14322 heartbeater.cc:503] Master 127.7.209.126:43463 was elected leader, sending a full tablet report...
I20260929 03:05:53.570593 14706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:53.580483 14790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:53.580499 14792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:53.580477 14789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:53.580749 14706 server_base.cc:1034] running on GCE node
I20260929 03:05:53.581537 14706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:53.582921 14706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:53.584090 14706 hybrid_clock.cc:648] HybridClock initialized: now 1790651153584067 us; error 37 us; skew 500 ppm
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:53.587126 14706 init.cc:377] Logged in from keytab as oryx/127.7.209.124@KRBTEST.COM (short username oryx)
I20260929 03:05:53.588115 14706 webserver.cc:460] Webserver started at http://127.7.209.124:40625/ using document root <none> and password file <none>
I20260929 03:05:53.588735 14706 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:53.588802 14706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:53.589035 14706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:53.591447 14706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data/instance:
uuid: "b399b817a1654b2bac7f203e6edc0243"
format_stamp: "Formatted at 2026-09-29 03:05:53 on dist-test-slave-bdps"
server_key: "fd59534ed339053c31da030495f3e8fe"
server_key_iv: "5c81717fc28b7cb7fee36848fe111724"
server_key_version: "encryptionkey@0"
I20260929 03:05:53.592166 14706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/wal/instance:
uuid: "b399b817a1654b2bac7f203e6edc0243"
format_stamp: "Formatted at 2026-09-29 03:05:53 on dist-test-slave-bdps"
server_key: "fd59534ed339053c31da030495f3e8fe"
server_key_iv: "5c81717fc28b7cb7fee36848fe111724"
server_key_version: "encryptionkey@0"
I20260929 03:05:53.596319 14706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 03:05:53.599020 14799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:53.600421 14706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:05:53.600538 14706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/wal
uuid: "b399b817a1654b2bac7f203e6edc0243"
format_stamp: "Formatted at 2026-09-29 03:05:53 on dist-test-slave-bdps"
server_key: "fd59534ed339053c31da030495f3e8fe"
server_key_iv: "5c81717fc28b7cb7fee36848fe111724"
server_key_version: "encryptionkey@0"
I20260929 03:05:53.600631 14706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:53.626362 14706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:53.630134 14706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:53.630334 14706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:53.638356 14706 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.124:35661
I20260929 03:05:53.638623 14851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.124:35661 every 8 connection(s)
I20260929 03:05:53.639513 14706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
I20260929 03:05:53.643044 14852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:53.643920 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 14706
I20260929 03:05:53.648082 14852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.209.124@KRBTEST.COM
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.124@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:53.659973 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.650249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.7.209.126:43463)
0929 03:05:53.650386 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.650390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.650588 (+ 198us) server_negotiation.cc:407] Connection header received
0929 03:05:53.650790 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.650793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.650847 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.650914 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.651770 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.652595 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.653854 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.654145 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.654237 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.656842 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.656845 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.656847 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.656874 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.658333 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.658758 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.658761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.658763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.658847 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.659118 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.659122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.659124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.659343 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.659419 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.659737 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.659833 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:05:53.660635 14855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.649981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:40740)
0929 03:05:53.650384 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.650440 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.650629 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.650981 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.650988 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.651479 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.651642 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.651649 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.652734 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.652737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.653732 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.653742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.654301 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.654306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.654412 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.654939 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.654955 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.656712 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.658423 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.658429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.658435 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.658685 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.658930 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.658933 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.658935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.659025 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.659437 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.659443 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.659553 (+ 110us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.659690 (+ 137us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.659860 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":111,"threads_started":1}
I20260929 03:05:53.660635 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.646147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34106 (local address 127.7.209.124:35661)
0929 03:05:53.646613 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.646631 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.646683 (+ 52us) server_negotiation.cc:407] Connection header received
0929 03:05:53.647417 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.647438 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.647868 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.648232 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.649088 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.650507 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.651742 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.652482 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.652636 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.655564 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.655571 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.655586 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.655613 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.657251 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.657739 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.657746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.657752 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.657882 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.658148 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.658151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.658153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.658609 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.658831 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.659115 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.659429 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":125,"threads_started":1}
I20260929 03:05:53.662446 14852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.124@KRBTEST.COM for oryx/127.7.209.125@KRBTEST.COM
I20260929 03:05:53.669780 14855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.662895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:44209 (local address 127.0.0.1:44142)
0929 03:05:53.662995 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.663008 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.663072 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.663641 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.663643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.663765 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.663852 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.663857 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.664726 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.664728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.665234 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.665240 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.665597 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.665599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.665744 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.666114 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.666127 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.667249 (+ 1122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.668684 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.668686 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.668688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.668851 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.669055 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.669056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.669058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.669116 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.669401 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.669403 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.669469 (+ 66us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.669537 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.669660 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 03:05:53.669857 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.663048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44142 (local address 127.7.209.125:44209)
0929 03:05:53.663344 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.663347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.663360 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:05:53.663400 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.663402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.663445 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.663624 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.663943 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.664638 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.665311 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.665529 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.665595 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.667396 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.667399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.667401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.667428 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.668596 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.668922 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.668925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.668926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.668997 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.669175 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.669177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.669178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.669342 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.669411 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.669577 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.669722 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":142,"threads_started":1}
I20260929 03:05:53.670575 14852 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.124@KRBTEST.COM for oryx/127.7.209.124@KRBTEST.COM
I20260929 03:05:53.677551 14855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.671009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35661 (local address 127.0.0.1:34122)
0929 03:05:53.671108 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.671121 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.671203 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.671381 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.671384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.671512 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.671564 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.671569 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.672609 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.672612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.673091 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.673098 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.673486 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.673489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.673556 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.673928 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.673940 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.674982 (+ 1042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.676344 (+ 1362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.676347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.676348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.676508 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.676729 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.676732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.676734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.676802 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.677097 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.677099 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.677181 (+ 82us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.677259 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.677381 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 03:05:53.677556 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.671043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34122 (local address 127.7.209.124:35661)
0929 03:05:53.671122 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.671125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.671168 (+ 43us) server_negotiation.cc:407] Connection header received
0929 03:05:53.671270 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.671273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.671306 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.671363 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.671670 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.672505 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.673184 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.673403 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.673481 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.675087 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.675090 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.675093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.675111 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.676252 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.676583 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.676586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.676587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.676648 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.676871 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.676874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.676876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.677019 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.677096 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.677310 (+ 214us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.677436 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":32}
I20260929 03:05:53.681162 14852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: Bootstrap starting.
I20260929 03:05:53.684507 14852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:53.685498 14852 log.cc:824] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:53.687928 14852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: No bootstrap required, opened a new log
I20260929 03:05:53.691483 14852 raft_consensus.cc:348] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } }
I20260929 03:05:53.691745 14852 raft_consensus.cc:374] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:53.691807 14852 raft_consensus.cc:729] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b399b817a1654b2bac7f203e6edc0243, State: Initialized, Role: FOLLOWER
I20260929 03:05:53.692337 14852 consensus_queue.cc:260] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } }
I20260929 03:05:53.692830 14858 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } } }
I20260929 03:05:53.692966 14858 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:05:53.693738 14852 sys_catalog.cc:564] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 03:05:53.699868 14869 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:05:53.699934 14869 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:05:53.703399 14706 master_runner.cc:428] Detected that this master b399b817a1654b2bac7f203e6edc0243 is joining an existing cluster
I20260929 03:05:53.703460 14706 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.209.124:35661
I20260929 03:05:53.714902 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 14193
I20260929 03:05:53.716369 14875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.706354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:40752)
0929 03:05:53.706917 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.706931 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.707244 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.707713 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.707717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.708076 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.708417 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.708427 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.709517 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.709520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.710975 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.710985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.711723 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.711726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.711913 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.712489 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.712500 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.712898 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.714739 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.714743 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.714745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.714981 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.715289 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.715291 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.715293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.715373 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.715826 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.715829 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.715916 (+ 87us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.716012 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.716136 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"mutex_wait_us":57,"thread_start_us":403,"threads_started":1}
I20260929 03:05:53.716441 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.706333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40752 (local address 127.7.209.126:43463)
0929 03:05:53.706468 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.706474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.707122 (+ 648us) server_negotiation.cc:407] Connection header received
0929 03:05:53.707391 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.707394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.707462 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.707547 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.708545 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.709393 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.711102 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.711430 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.711500 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.713022 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.713026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.713028 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.713051 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.714640 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.715086 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.715090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.715092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.715191 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.715468 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.715471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.715473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.715685 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.715753 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.716119 (+ 366us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.716280 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 03:05:53.724620 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:39031
--local_ip_for_outbound_sockets=127.7.209.65
--tserver_master_addrs=127.7.209.126:43463
--webserver_port=46805
--webserver_interface=127.7.209.65
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab
--principal=oryx/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.209.126:43463,127.7.209.125:44209
--tserver_master_addrs=127.7.209.126:43463,127.7.209.125:44209,127.7.209.124:35661 with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651152, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.126@KRBTEST.COM for oryx/127.7.209.124@KRBTEST.COM
I20260929 03:05:53.734264 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.707176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:44209 (local address 127.0.0.1:44150)
0929 03:05:53.707833 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.707846 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.708100 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.708491 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.708494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.708774 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.708971 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.708979 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.710000 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.710003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.711062 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.711070 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.711736 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.711739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.712234 (+ 495us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.730100 (+ 17866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.730114 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.730554 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.732627 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.732630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.732632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.732868 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.733169 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.733172 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.733174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.733251 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.733683 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.733686 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.733782 (+ 96us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.733861 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.734007 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":72,"thread_start_us":293,"threads_started":1}
I20260929 03:05:53.734262 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.720248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35661 (local address 127.0.0.1:34140)
0929 03:05:53.720370 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.720385 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.720460 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.724499 (+ 4039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.724503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.724746 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.724872 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.724880 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.725909 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.725913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.726829 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.726840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.727413 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.727417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.727515 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.728026 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.728040 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.729715 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.732226 (+ 2511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.732229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.732231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.732484 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.732801 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.732804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.732806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.732898 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.733721 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.733725 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.733835 (+ 110us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.733920 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.734073 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 03:05:53.735394 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.707225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44150 (local address 127.7.209.125:44209)
0929 03:05:53.707661 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.707665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.708040 (+ 375us) server_negotiation.cc:407] Connection header received
0929 03:05:53.708229 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.708232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.708295 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.708353 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.709096 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.709861 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.711177 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.711507 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.711568 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.730765 (+ 19197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.730770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.730774 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.730808 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.732494 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.732987 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.732991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.732993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.733078 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.733339 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.733342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.733344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.733552 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.735018 (+ 1466us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.735091 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.735232 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335}
I20260929 03:05:53.736212 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.723856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34140 (local address 127.7.209.124:35661)
0929 03:05:53.724179 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.724189 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.724203 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:05:53.724282 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.724285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.724339 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.724442 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.725000 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.725781 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.726956 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.727301 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.727406 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.729895 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.729899 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.729902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.729928 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.732085 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.732606 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.732609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.732611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.732692 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.733011 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.733014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.733016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.733243 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.734788 (+ 1545us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.734885 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.736024 (+ 1139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":101,"threads_started":1}
I20260929 03:05:53.737756 14131 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.209.124:35661: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b399b817a1654b2bac7f203e6edc0243"
member_type: NON_VOTER
last_known_addr {
host: "127.7.209.124"
port: 35661
}
attrs {
promote: true
}
}
dest_uuid: "073adfa114bf4d518ffcce398e022987"
cas_config_opid_index: 5
I20260929 03:05:53.738600 14131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } }
I20260929 03:05:53.739868 14692 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b399b817a1654b2bac7f203e6edc0243 (127.7.209.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:53.741039 14826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:53.741205 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.706634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34132 (local address 127.7.209.124:35661)
0929 03:05:53.706740 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.706745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.707880 (+ 1135us) server_negotiation.cc:407] Connection header received
0929 03:05:53.708027 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.708030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.708066 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.708109 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.708884 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.709661 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.711607 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.711971 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.712069 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.733381 (+ 21312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.733384 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.733387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.733409 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.735531 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.737440 (+ 1909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.737443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.737445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.737522 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.740263 (+ 2741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.740266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.740268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.740538 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.740602 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.740956 (+ 354us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.741060 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":42}
I20260929 03:05:53.741662 14877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.706423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35661 (local address 127.0.0.1:34132)
0929 03:05:53.707690 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.707706 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.707934 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.708320 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.708323 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.708638 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.708773 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.708781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.709782 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.709785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.711485 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.711495 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.712078 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.712082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.712204 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.730842 (+ 18638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.730857 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.731292 (+ 435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.735642 (+ 4350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.735645 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.735647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.735865 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.740065 (+ 4200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.740067 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.740069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.740172 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.740728 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.740730 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.740822 (+ 92us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.741357 (+ 535us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.741481 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1181,"mutex_wait_us":78,"thread_start_us":348,"threads_started":1}
I20260929 03:05:53.741571 14692 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.209.124:35661
I20260929 03:05:53.741992 14691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:53.742107 14691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:53.742658 14826 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 FOLLOWER]: Refusing update from remote peer 073adfa114bf4d518ffcce398e022987: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 03:05:53.742703 14706 master.cc:537] Master@127.7.209.124:35661 shutting down...
I20260929 03:05:53.743757 14691 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 03:05:53.745441 14116 consensus_peers.cc:487] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 -> Peer b399b817a1654b2bac7f203e6edc0243 (127.7.209.124:35661): Couldn't send request to peer b399b817a1654b2bac7f203e6edc0243. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 03:05:53.747516 14688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.740063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:44209 (local address 127.0.0.1:44152)
0929 03:05:53.740799 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.740813 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.740890 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.741459 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.741462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.741479 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:53.741556 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.741561 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.744645 (+ 3084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.744649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.746173 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.746184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.747121 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.747125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.747213 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.747270 (+ 57us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.747343 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.747408 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671}
I20260929 03:05:53.747529 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.740357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44152 (local address 127.7.209.125:44209)
0929 03:05:53.740714 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.740718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.740961 (+ 243us) server_negotiation.cc:407] Connection header received
0929 03:05:53.740998 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.741001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.741064 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.741152 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:53.741712 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.742425 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.746309 (+ 3884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.747004 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.747107 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.747280 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.747352 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.747413 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271}
I20260929 03:05:53.748245 14739 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 LEARNER]: Refusing update from remote peer 073adfa114bf4d518ffcce398e022987: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 03:05:53.748636 14691 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 03:05:53.750816 14739 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b399b817a1654b2bac7f203e6edc0243 (127.7.209.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:53.752974 14773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 073adfa114bf4d518ffcce398e022987. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:53.753088 14773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: This master's current role is: LEARNER
I20260929 03:05:53.753159 14773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: NON_VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: true } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:53.753228 14773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: This master's current role is: LEARNER
I20260929 03:05:53.753865 14691 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987: attempting to promote NON_VOTER 1b84e1dc781d456c90c2ceefed504316 to VOTER
I20260929 03:05:53.754316 14691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } }
I20260929 03:05:53.755158 14706 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:53.755472 14706 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:53.755568 14706 tablet_replica.cc:330] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: stopping tablet replica
I20260929 03:05:53.755697 14743 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 LEARNER]: Refusing update from remote peer 073adfa114bf4d518ffcce398e022987: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260929 03:05:53.756192 14694 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260929 03:05:53.757817 14694 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1b84e1dc781d456c90c2ceefed504316 (127.7.209.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:53.759279 14694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:53.759378 14694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:53.759472 14694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:53.759533 14694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: This master's current role is: LEADER
W20260929 03:05:53.759819 14116 consensus_peers.cc:487] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 -> Peer b399b817a1654b2bac7f203e6edc0243 (127.7.209.124:35661): Couldn't send request to peer b399b817a1654b2bac7f203e6edc0243. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 03:05:53.760128 14743 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 1b84e1dc781d456c90c2ceefed504316 (127.7.209.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:53.761732 14881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:53.761837 14881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:05:53.774282 14706 master.cc:552] Master@127.7.209.124:35661 shutdown complete.
I20260929 03:05:53.774371 14706 master_runner.cc:305] Clearing existing system tablet
I20260929 03:05:53.774654 14706 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:53.774726 14706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:53.776196 14706 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260929 03:05:53.777216 14883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:53.777527 14706 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:53.777750 14706 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:53.777791 14706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/wal
uuid: "b399b817a1654b2bac7f203e6edc0243"
format_stamp: "Formatted at 2026-09-29 03:05:53 on dist-test-slave-bdps"
server_key: "fd59534ed339053c31da030495f3e8fe"
server_key_iv: "5c81717fc28b7cb7fee36848fe111724"
server_key_version: "encryptionkey@0"
I20260929 03:05:53.780259 14706 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:53.783546 14706 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 03:05:53.783644 14706 log.cc:1196] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 03:05:53.784039 14706 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: Deleting consensus metadata
I20260929 03:05:53.784211 14706 master_runner.cc:315] Copying system tablet from 127.7.209.126:43463
I20260929 03:05:53.785578 14706 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: tablet copy: Beginning tablet copy session from remote peer at address 127.7.209.126:43463
I20260929 03:05:53.798154 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.788476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:40766)
0929 03:05:53.788977 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.788997 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.789151 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.789461 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.789464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.789776 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.789953 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.789963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.791080 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.791084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.792128 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.792142 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.792797 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.792802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.792919 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.793600 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:53.793649 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:53.794271 (+ 622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:53.796440 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.796444 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.796447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.796722 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.797009 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:53.797012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:53.797014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:53.797102 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:53.797514 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:53.797517 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:53.797671 (+ 154us) client_negotiation.cc:769] Sending connection context
0929 03:05:53.797772 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 03:05:53.797938 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":224,"threads_started":1}
I20260929 03:05:53.798154 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.788612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40766 (local address 127.7.209.126:43463)
0929 03:05:53.788791 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.788797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.789123 (+ 326us) server_negotiation.cc:407] Connection header received
0929 03:05:53.789257 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.789261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.789333 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.789434 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.790090 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.790956 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.792273 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.792672 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.792783 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:53.794441 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:53.794446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:53.794449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:53.794487 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:53.796341 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.796821 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.796824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.796826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.796926 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:53.797186 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:53.797189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:53.797191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:53.797428 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:53.797525 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:53.797855 (+ 330us) server_negotiation.cc:299] Negotiation successful
0929 03:05:53.797991 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 03:05:53.798902 14151 tablet_copy_service.cc:140] P 073adfa114bf4d518ffcce398e022987: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b399b817a1654b2bac7f203e6edc0243 ({username='oryx', principal='oryx/127.7.209.124@KRBTEST.COM'} at 127.0.0.1:40766)
I20260929 03:05:53.799027 14151 tablet_copy_service.cc:161] P 073adfa114bf4d518ffcce398e022987: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b399b817a1654b2bac7f203e6edc0243 at {username='oryx', principal='oryx/127.7.209.124@KRBTEST.COM'} at 127.0.0.1:40766: session id = b399b817a1654b2bac7f203e6edc0243-00000000000000000000000000000000
I20260929 03:05:53.800357 14151 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 03:05:53.801992 14706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 03:05:53.804925 14706 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: tablet copy: Starting download of 0 data blocks...
I20260929 03:05:53.805157 14706 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: tablet copy: Starting download of 1 WAL segments...
I20260929 03:05:53.807093 14706 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 03:05:53.808811 14456 heartbeater.cc:503] Master 127.7.209.126:43463 was elected leader, sending a full tablet report...
I20260929 03:05:53.809407 14151 tablet_copy_service.cc:342] P 073adfa114bf4d518ffcce398e022987: Request end of tablet copy session b399b817a1654b2bac7f203e6edc0243-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.209.124@KRBTEST.COM'} at 127.0.0.1:40766
I20260929 03:05:53.809506 14151 tablet_copy_service.cc:433] P 073adfa114bf4d518ffcce398e022987: ending tablet copy session b399b817a1654b2bac7f203e6edc0243-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b399b817a1654b2bac7f203e6edc0243
I20260929 03:05:53.818745 14706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:53.820492 14890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:53.820575 14893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:53.821017 14891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:53.821028 14706 server_base.cc:1034] running on GCE node
I20260929 03:05:53.821352 14706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:53.821715 14706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:53.822855 14706 hybrid_clock.cc:648] HybridClock initialized: now 1790651153822842 us; error 33 us; skew 500 ppm
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:53.824679 14706 init.cc:377] Logged in from keytab as oryx/127.7.209.124@KRBTEST.COM (short username oryx)
I20260929 03:05:53.825143 14706 webserver.cc:460] Webserver started at http://127.7.209.124:35083/ using document root <none> and password file <none>
I20260929 03:05:53.825337 14706 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:53.825382 14706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:53.826900 14706 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:53.828001 14900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:53.828289 14706 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:53.828362 14706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/wal
uuid: "b399b817a1654b2bac7f203e6edc0243"
format_stamp: "Formatted at 2026-09-29 03:05:53 on dist-test-slave-bdps"
server_key: "fd59534ed339053c31da030495f3e8fe"
server_key_iv: "5c81717fc28b7cb7fee36848fe111724"
server_key_version: "encryptionkey@0"
I20260929 03:05:53.828452 14706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:53.863875 14706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:53.866261 14706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:53.873699 14706 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.124:35661
I20260929 03:05:53.874397 14952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.124:35661 every 8 connection(s)
I20260929 03:05:53.874614 14706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
I20260929 03:05:53.876446 14953 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:05:53.877329 14953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: Bootstrap starting.
I20260929 03:05:53.887938 14953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:05:53.888255 14953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: Bootstrap complete.
I20260929 03:05:53.888875 14953 raft_consensus.cc:348] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } }
I20260929 03:05:53.889016 14953 raft_consensus.cc:729] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b399b817a1654b2bac7f203e6edc0243, State: Initialized, Role: LEARNER
I20260929 03:05:53.889133 14953 consensus_queue.cc:260] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } }
I20260929 03:05:53.889678 14953 sys_catalog.cc:564] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:53.889649 14956 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:53.889835 14956 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: This master's current role is: LEARNER
I20260929 03:05:53.894336 14967 catalog_manager.cc:1259] Loaded cluster ID: 8689b40822d54e7ab5d5845ac4e6d649
I20260929 03:05:53.894390 14967 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: loading cluster ID for follower catalog manager: success
I20260929 03:05:53.896034 14967 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: acquiring CA information for follower catalog manager: success
I20260929 03:05:53.897287 14967 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 03:05:53.899539 14879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:53.899814 14879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:53.899887 14879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:53.904081 14879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:53.904137 14879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:53.904218 14879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:53.909891 14879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab
--principal=oryx/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:39031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=46805
--tserver_master_addrs=127.7.209.126:43463,127.7.209.125:44209,127.7.209.124:35661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:53.911201 14879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:53.912446 14879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:53.924747 14973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:53.924918 14879 server_base.cc:1034] running on GCE node
W20260929 03:05:53.924796 14972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:53.924737 14975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:53.925750 14879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:53.926597 14879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:53.927786 14879 hybrid_clock.cc:648] HybridClock initialized: now 1790651153927764 us; error 35 us; skew 500 ppm
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:53.931694 14879 init.cc:377] Logged in from keytab as oryx/127.7.209.65@KRBTEST.COM (short username oryx)
I20260929 03:05:53.932776 14879 webserver.cc:460] Webserver started at http://127.7.209.65:46805/ using document root <none> and password file <none>
I20260929 03:05:53.933422 14879 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:53.933501 14879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:53.938393 14879 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260929 03:05:53.940930 14982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:53.942466 14879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 03:05:53.942575 14879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "e244ba09d89d4662a64088da114ca347"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "eeb2fefe4aa870d6bb93074501e7a442"
server_key_iv: "51116ae655a8ea3bdc6339449aedddc1"
server_key_version: "encryptionkey@0"
I20260929 03:05:53.943343 14879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:53.962715 14879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:53.966228 14879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:53.966459 14879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:53.967373 14879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:53.968662 14879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:53.968722 14879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:53.968780 14879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:53.968806 14879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:53.986783 14879 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:39031
I20260929 03:05:53.986872 15095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:39031 every 8 connection(s)
I20260929 03:05:53.988130 14879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:53.988574 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 14879
I20260929 03:05:53.988823 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 14327
I20260929 03:05:53.997097 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:34659
--local_ip_for_outbound_sockets=127.7.209.66
--tserver_master_addrs=127.7.209.126:43463
--webserver_port=40047
--webserver_interface=127.7.209.66
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab
--principal=oryx/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.209.126:43463,127.7.209.125:44209
--tserver_master_addrs=127.7.209.126:43463,127.7.209.125:44209,127.7.209.124:35661 with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.124@KRBTEST.COM
Sep 29 03:05:53 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651153, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.125@KRBTEST.COM
I20260929 03:05:54.009709 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.993767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:41817 (local address 127.7.209.125:44209)
0929 03:05:53.993913 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.993918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.993935 (+ 17us) server_negotiation.cc:407] Connection header received
0929 03:05:53.994642 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.994646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.994724 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.994783 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.996849 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.997729 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.000020 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.000338 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.000416 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.004882 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.004886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.004889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.004922 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.006763 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.007540 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.007544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.007546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.007674 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.008107 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.008110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.008111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.008400 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.008482 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.009329 (+ 847us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.009476 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 03:05:54.009709 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.993114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:33965 (local address 127.7.209.126:43463)
0929 03:05:53.993265 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.993270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.993289 (+ 19us) server_negotiation.cc:407] Connection header received
0929 03:05:53.994653 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.994657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.994724 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.994787 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.996848 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.997729 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.999749 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.000082 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.000165 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.004882 (+ 4717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.004886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.004889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.004923 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.007422 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.007909 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.007913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.007914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.008006 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.008313 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.008317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.008319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.008538 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.008602 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.009331 (+ 729us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.009476 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 03:05:54.009959 15102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.992062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:50431 (local address 127.7.209.124:35661)
0929 03:05:53.993962 (+ 1900us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:53.993965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:53.993979 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:05:53.994731 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:53.994734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:53.994793 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:53.994876 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:53.996820 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.997862 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.999743 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.000080 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.000175 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.004679 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.004685 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.004688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.004727 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.006720 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.007547 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.007551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.007553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.007686 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.008124 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.008128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.008129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.008400 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.008499 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.009344 (+ 845us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.009496 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1820,"thread_start_us":124,"threads_started":1}
I20260929 03:05:54.011673 15103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.991319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.65:33965)
0929 03:05:53.992901 (+ 1582us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.992966 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.994381 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.995148 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.995162 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.995824 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.996654 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.996671 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.997978 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.997981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.999610 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.999621 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.000356 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.000360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.000647 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.001567 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:54.001580 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:54.004697 (+ 3117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:54.007554 (+ 2857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.007557 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.007559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.007799 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.008112 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.008114 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.008116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.008215 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.008667 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:54.008670 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:54.008998 (+ 328us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.009898 (+ 900us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.010367 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1415,"mutex_wait_us":250,"spinlock_wait_cycles":237056,"thread_start_us":144,"threads_started":1}
I20260929 03:05:54.011685 15100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.991247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35661 (local address 127.7.209.65:50431)
0929 03:05:53.992901 (+ 1654us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.992965 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.994362 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.995148 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.995158 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.995812 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.996648 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.996676 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.997982 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.997985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.999603 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.999614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.000356 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.000360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.000645 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.001722 (+ 1077us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:54.001737 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:54.004389 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:54.006868 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.006876 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.006900 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.007423 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.007800 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.007804 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.007806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.008012 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.008545 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:54.008549 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:54.009000 (+ 451us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.010181 (+ 1181us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.010451 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1388,"mutex_wait_us":451,"spinlock_wait_cycles":174848,"thread_start_us":228,"threads_started":1}
I20260929 03:05:54.011678 15101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:53.992549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:44209 (local address 127.7.209.65:41817)
0929 03:05:53.993526 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:53.993538 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:53.994384 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:53.995148 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:53.995158 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:53.995831 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:53.996665 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.996671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:53.998180 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:53.998184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:53.999901 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:53.999910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.000591 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.000595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.000717 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.001527 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:54.001551 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:54.004697 (+ 3146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:54.006938 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.006941 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.006943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.007415 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.007795 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.007798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.007799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.008001 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.008535 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:54.008543 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:54.008998 (+ 455us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.011193 (+ 2195us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.011337 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"mutex_wait_us":263,"spinlock_wait_cycles":25856,"thread_start_us":85,"threads_started":1}
I20260929 03:05:54.013520 15096 heartbeater.cc:347] Connected to a master server at 127.7.209.124:35661
I20260929 03:05:54.013583 15098 heartbeater.cc:347] Connected to a master server at 127.7.209.125:44209
I20260929 03:05:54.013666 15097 heartbeater.cc:347] Connected to a master server at 127.7.209.126:43463
I20260929 03:05:54.013867 15097 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:54.013868 15098 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:54.013895 15096 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:54.014608 15096 heartbeater.cc:511] Master 127.7.209.124:35661 requested a full tablet report, sending...
I20260929 03:05:54.014608 15097 heartbeater.cc:511] Master 127.7.209.126:43463 requested a full tablet report, sending...
I20260929 03:05:54.014608 15098 heartbeater.cc:511] Master 127.7.209.125:44209 requested a full tablet report, sending...
I20260929 03:05:54.016475 14730 ts_manager.cc:194] Registered new tserver with Master: e244ba09d89d4662a64088da114ca347 (127.7.209.65:39031)
I20260929 03:05:54.016475 14917 ts_manager.cc:194] Registered new tserver with Master: e244ba09d89d4662a64088da114ca347 (127.7.209.65:39031)
I20260929 03:05:54.017283 14131 ts_manager.cc:194] Re-registered known tserver with Master: e244ba09d89d4662a64088da114ca347 (127.7.209.65:39031)
I20260929 03:05:54.017970 14131 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:33965
I20260929 03:05:54.046383 14590 heartbeater.cc:503] Master 127.7.209.126:43463 was elected leader, sending a full tablet report...
W20260929 03:05:54.133157 15104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:54.133496 15104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:54.133564 15104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:54.137732 15104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:54.137795 15104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:54.137882 15104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:54.143324 15104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab
--principal=oryx/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:34659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=40047
--tserver_master_addrs=127.7.209.126:43463,127.7.209.125:44209,127.7.209.124:35661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:54.144769 15104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:54.146315 15104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:54.158278 15110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:54.158284 15112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:54.158281 15109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:54.158610 15104 server_base.cc:1034] running on GCE node
I20260929 03:05:54.159170 15104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:54.159943 15104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:54.161104 15104 hybrid_clock.cc:648] HybridClock initialized: now 1790651154161077 us; error 47 us; skew 500 ppm
Sep 29 03:05:54 dist-test-slave-bdps krb5kdc[14086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651154, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:54.164692 15104 init.cc:377] Logged in from keytab as oryx/127.7.209.66@KRBTEST.COM (short username oryx)
I20260929 03:05:54.166031 15104 webserver.cc:460] Webserver started at http://127.7.209.66:40047/ using document root <none> and password file <none>
I20260929 03:05:54.166774 15104 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:54.166873 15104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:54.171823 15104 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260929 03:05:54.174258 15119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:54.175743 15104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:54.175858 15104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "8b38a7e3aef14c47826af2d89ef37697"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "7c9d0770ac307584c6e7e408fa953b0e"
server_key_iv: "f02dde08d823245e76f6a995848106ca"
server_key_version: "encryptionkey@0"
I20260929 03:05:54.176484 15104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:54.205910 15104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:54.209883 15104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:54.210100 15104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:54.210860 15104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:54.212116 15104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:54.212178 15104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:54.212242 15104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:54.212268 15104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:54.225664 15104 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:34659
I20260929 03:05:54.225698 15232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:34659 every 8 connection(s)
I20260929 03:05:54.227007 15104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:54.236379 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 15104
I20260929 03:05:54.236599 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 14461
Sep 29 03:05:54 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651154, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
Sep 29 03:05:54 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651154, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.124@KRBTEST.COM
Sep 29 03:05:54 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651154, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.125@KRBTEST.COM
I20260929 03:05:54.246333 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:37215
--local_ip_for_outbound_sockets=127.7.209.67
--tserver_master_addrs=127.7.209.126:43463
--webserver_port=35023
--webserver_interface=127.7.209.67
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab
--principal=oryx/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.209.126:43463,127.7.209.125:44209
--tserver_master_addrs=127.7.209.126:43463,127.7.209.125:44209,127.7.209.124:35661 with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:05:54.246980 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.230578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:37367 (local address 127.7.209.126:43463)
0929 03:05:54.230700 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.230704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.231750 (+ 1046us) server_negotiation.cc:407] Connection header received
0929 03:05:54.232657 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.232661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.232724 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.232781 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.234950 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.235813 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.237905 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.238238 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.238311 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.242366 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.242370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.242372 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.242405 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.244647 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.245119 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.245122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.245124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.245234 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.245548 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.245551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.245553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.245815 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.245900 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.246676 (+ 776us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.246806 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 03:05:54.246981 15102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.230288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:58667 (local address 127.7.209.124:35661)
0929 03:05:54.230472 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.230477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.231274 (+ 797us) server_negotiation.cc:407] Connection header received
0929 03:05:54.232660 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.232665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.232732 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.232783 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.234943 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.235806 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.237671 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.238012 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.238098 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.242243 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.242247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.242250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.242282 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.244131 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.244916 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.244921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.244924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.245029 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.245461 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.245465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.245466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.245749 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.245832 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.246644 (+ 812us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.246809 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260929 03:05:54.246996 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.231319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:54711 (local address 127.7.209.125:44209)
0929 03:05:54.231453 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.231458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.232032 (+ 574us) server_negotiation.cc:407] Connection header received
0929 03:05:54.232656 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.232668 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.232724 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.232781 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.234949 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.235821 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.240710 (+ 4889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.240961 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.241055 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.243614 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.243618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.243622 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.243662 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.245210 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.245720 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.245724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.245726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.245821 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.246103 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.246107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.246108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.246336 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.246422 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.246731 (+ 309us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.246853 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 03:05:54.249060 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.231186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:44209 (local address 127.7.209.66:54711)
0929 03:05:54.231864 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:54.231874 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:54.232394 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:54.233236 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:54.233245 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:54.233921 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:54.234778 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.234793 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.235981 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.235984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.240573 (+ 4589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.240591 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.241253 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.241256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.241321 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.241970 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:54.241982 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:54.243399 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:54.245321 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.245324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.245327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.245564 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.245919 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.245922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.245924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.246003 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.246476 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:54.246479 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:54.246552 (+ 73us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.247190 (+ 638us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.247734 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":208,"spinlock_wait_cycles":2304,"thread_start_us":153,"threads_started":1}
I20260929 03:05:54.249107 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.229987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.66:37367)
0929 03:05:54.231562 (+ 1575us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:54.231574 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:54.232394 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:54.233236 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:54.233245 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:54.233921 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:54.234783 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.234793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.235981 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.235984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.237771 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.237782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.238469 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.238473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.238623 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.239422 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:54.239443 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:54.242231 (+ 2788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:54.244768 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.244771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.244774 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.245010 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.245339 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.245342 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.245343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.245456 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.245956 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:54.245959 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:54.246336 (+ 377us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.247191 (+ 855us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.247671 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1496,"mutex_wait_us":276,"spinlock_wait_cycles":77184,"thread_start_us":226,"threads_started":1}
I20260929 03:05:54.249109 15237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.229986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35661 (local address 127.7.209.66:58667)
0929 03:05:54.230985 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:54.231043 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:54.232394 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:54.233235 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:54.233245 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:54.233921 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:54.234775 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.234793 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.235979 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.235983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.237513 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.237524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.238313 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.238317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.238609 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.239540 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:54.239554 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:54.242022 (+ 2468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:54.244266 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.244274 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.244298 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.244795 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.245153 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.245156 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.245159 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.245358 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.245900 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:54.245907 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:54.246336 (+ 429us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.247232 (+ 896us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.247800 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":376,"spinlock_wait_cycles":20864,"thread_start_us":146,"threads_started":1}
I20260929 03:05:54.250619 15235 heartbeater.cc:347] Connected to a master server at 127.7.209.125:44209
I20260929 03:05:54.250617 15234 heartbeater.cc:347] Connected to a master server at 127.7.209.126:43463
I20260929 03:05:54.250908 15235 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:54.250907 15234 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:54.251520 15235 heartbeater.cc:511] Master 127.7.209.125:44209 requested a full tablet report, sending...
I20260929 03:05:54.251519 15234 heartbeater.cc:511] Master 127.7.209.126:43463 requested a full tablet report, sending...
I20260929 03:05:54.252120 15233 heartbeater.cc:347] Connected to a master server at 127.7.209.124:35661
I20260929 03:05:54.252205 15233 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:54.252349 15233 heartbeater.cc:511] Master 127.7.209.124:35661 requested a full tablet report, sending...
I20260929 03:05:54.252765 14131 ts_manager.cc:194] Re-registered known tserver with Master: 8b38a7e3aef14c47826af2d89ef37697 (127.7.209.66:34659)
I20260929 03:05:54.252935 14730 ts_manager.cc:194] Registered new tserver with Master: 8b38a7e3aef14c47826af2d89ef37697 (127.7.209.66:34659)
I20260929 03:05:54.253140 14917 ts_manager.cc:194] Registered new tserver with Master: 8b38a7e3aef14c47826af2d89ef37697 (127.7.209.66:34659)
I20260929 03:05:54.253597 14131 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:37367
I20260929 03:05:54.344381 15102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.340455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34150 (local address 127.7.209.124:35661)
0929 03:05:54.340608 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.340613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.340816 (+ 203us) server_negotiation.cc:407] Connection header received
0929 03:05:54.340996 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.340998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.341065 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.341165 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:54.341460 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.342086 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.343229 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.343805 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.343915 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.344106 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.344143 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.344254 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:05:54.344385 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.340378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35661 (local address 127.0.0.1:34150)
0929 03:05:54.340709 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:54.340723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:54.340877 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:54.341190 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:54.341194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:54.341216 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:54.341351 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.341357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.342206 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.342211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.343101 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.343113 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.343927 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.343930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.344027 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.344053 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.344118 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.344210 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":130,"threads_started":1}
I20260929 03:05:54.345351 14927 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260929 03:05:54.345855 14956 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 073adfa114bf4d518ffcce398e022987. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:54.346017 14956 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: This master's current role is: LEARNER
W20260929 03:05:54.378135 15240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:54.378450 15240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:54.378495 15240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:54.382682 15240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:54.382754 15240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:54.382815 15240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:54.388438 15240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab
--principal=oryx/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:37215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=35023
--tserver_master_addrs=127.7.209.126:43463,127.7.209.125:44209,127.7.209.124:35661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:54.389832 15240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:54.391206 15240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:54.402736 15247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:54.402736 15248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:54.402770 15250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:54.403198 15240 server_base.cc:1034] running on GCE node
I20260929 03:05:54.403744 15240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:54.404615 15240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:54.405800 15240 hybrid_clock.cc:648] HybridClock initialized: now 1790651154405777 us; error 40 us; skew 500 ppm
Sep 29 03:05:54 dist-test-slave-bdps krb5kdc[14086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651154, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:54.409212 15240 init.cc:377] Logged in from keytab as oryx/127.7.209.67@KRBTEST.COM (short username oryx)
I20260929 03:05:54.410429 15240 webserver.cc:460] Webserver started at http://127.7.209.67:35023/ using document root <none> and password file <none>
I20260929 03:05:54.411144 15240 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:54.411249 15240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:54.416390 15240 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 03:05:54.419001 15257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:54.420516 15240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:05:54.420637 15240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "c49acfe4441c4b0a84aa8e41ac0cc054"
format_stamp: "Formatted at 2026-09-29 03:05:52 on dist-test-slave-bdps"
server_key: "8f0434620c86bc717f11f76c6b254275"
server_key_iv: "56ee47d1db72c4226688cb3738395292"
server_key_version: "encryptionkey@0"
I20260929 03:05:54.421304 15240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:54.436707 15240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:54.439935 15240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:54.440109 15240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:54.440748 15240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:05:54.441736 15240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:54.441787 15240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:54.441843 15240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:54.441869 15240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:54.452685 15370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:37215 every 8 connection(s)
I20260929 03:05:54.452685 15240 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:37215
I20260929 03:05:54.454185 15240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:54.459008 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 15240
Sep 29 03:05:54 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651154, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
Sep 29 03:05:54 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651154, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.125@KRBTEST.COM
Sep 29 03:05:54 dist-test-slave-bdps krb5kdc[14086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651154, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.124@KRBTEST.COM
I20260929 03:05:54.473297 15384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.462121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40776 (local address 127.7.209.126:43463)
0929 03:05:54.462907 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.462921 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.462935 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:05:54.462972 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.462975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.463021 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.463117 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.464094 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.464867 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.466759 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.467118 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.467608 (+ 490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.468912 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.468916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.468919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.468947 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.471122 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.471610 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.471613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.471615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.471707 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.472002 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.472005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.472006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.472190 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.472881 (+ 691us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.472953 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.473092 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":74,"threads_started":1}
I20260929 03:05:54.473493 15102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.456488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:35989 (local address 127.7.209.124:35661)
0929 03:05:54.456624 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.456629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.457596 (+ 967us) server_negotiation.cc:407] Connection header received
0929 03:05:54.458916 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.458920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.458991 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.459048 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.465010 (+ 5962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.465790 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.466886 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.467238 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.467346 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.469989 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.469994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.469997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.470029 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.471584 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.472061 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.472065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.472067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.472161 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.472469 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.472473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.472475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.472709 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.472799 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.473197 (+ 398us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.473320 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 03:05:54.474467 15375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.456876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.67:47017)
0929 03:05:54.457422 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:54.457481 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:54.458656 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:54.459471 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:54.459480 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:54.460126 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:54.460954 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.460971 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.462105 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.462109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.463456 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.463466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.464204 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.464207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.464472 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.465271 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:54.465292 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:54.468136 (+ 2844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:54.470721 (+ 2585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.470728 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.470751 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.471122 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.471450 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.471453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.471455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.471665 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.472142 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:54.472149 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:54.472547 (+ 398us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.472777 (+ 230us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.473194 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":150,"spinlock_wait_cycles":25728,"thread_start_us":130,"threads_started":1}
I20260929 03:05:54.475353 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.456290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:47017 (local address 127.7.209.126:43463)
0929 03:05:54.456479 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.456484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.457596 (+ 1112us) server_negotiation.cc:407] Connection header received
0929 03:05:54.458905 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.458909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.458970 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.459027 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.461127 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.461943 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.463593 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.463956 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.464062 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.468353 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.468357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.468360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.468386 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.470587 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.471243 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.471247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.471249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.471349 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.471765 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.471769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.471770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.472004 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.472088 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.475081 (+ 2993us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.475210 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 03:05:54.475719 15376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.456877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35661 (local address 127.7.209.67:35989)
0929 03:05:54.457422 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:54.457481 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:54.458651 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:54.459471 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:54.459480 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:54.460180 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:54.464855 (+ 4675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.464864 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.465925 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.465929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.466747 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.466757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.467478 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.467482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.467593 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.468027 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:54.468041 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:54.469833 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:54.471702 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.471705 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.471707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.471951 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.472266 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.472269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.472271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.472359 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.472844 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:54.472847 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:54.472936 (+ 89us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.473032 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.475560 (+ 2528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":2430,"spinlock_wait_cycles":98688,"thread_start_us":134,"threads_started":1}
I20260929 03:05:54.476698 15371 heartbeater.cc:347] Connected to a master server at 127.7.209.124:35661
I20260929 03:05:54.476703 15372 heartbeater.cc:347] Connected to a master server at 127.7.209.126:43463
I20260929 03:05:54.477010 15372 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:54.477761 15372 heartbeater.cc:511] Master 127.7.209.126:43463 requested a full tablet report, sending...
I20260929 03:05:54.477010 15371 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:54.478511 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.458219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:40529 (local address 127.7.209.125:44209)
0929 03:05:54.458339 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.458342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.458356 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:05:54.458907 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.458910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.458970 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.459027 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.461135 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.461944 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.463574 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.463910 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.463983 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.471493 (+ 7510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.471498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.471501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.471532 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.473192 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.474142 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.474145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.474147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.474239 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.474536 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.474539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.474541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.474789 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.474886 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.478199 (+ 3313us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.478337 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 03:05:54.478691 15371 heartbeater.cc:511] Master 127.7.209.124:35661 requested a full tablet report, sending...
I20260929 03:05:54.479271 15377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.458039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:44209 (local address 127.7.209.67:40529)
0929 03:05:54.458276 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:54.458290 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:54.458652 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:54.459471 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:54.459480 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:54.460126 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:54.460954 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.460970 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.462336 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.462339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.463451 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.463461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.464204 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.464208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.464473 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.466122 (+ 1649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:54.466137 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:54.471335 (+ 5198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:54.473313 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.473316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.473318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.474041 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.474343 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:54.474346 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:54.474349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:54.474439 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:54.474890 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:54.474893 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:54.477728 (+ 2835us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.478653 (+ 925us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.478812 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":175,"spinlock_wait_cycles":1920,"thread_start_us":88,"threads_started":1}
I20260929 03:05:54.479643 14917 ts_manager.cc:194] Registered new tserver with Master: c49acfe4441c4b0a84aa8e41ac0cc054 (127.7.209.67:37215)
I20260929 03:05:54.480934 15383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.461820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44162 (local address 127.7.209.125:44209)
0929 03:05:54.462095 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.462099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.463479 (+ 1380us) server_negotiation.cc:407] Connection header received
0929 03:05:54.463704 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.463707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.463770 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.463836 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.465329 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.466115 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.467252 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.468058 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.468136 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.476854 (+ 8718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.476858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.476861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.476887 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.479239 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.479648 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.479651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.479653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.479742 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.480012 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.480016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.480017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.480275 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.480366 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.480650 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.480782 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":101,"threads_started":1}
I20260929 03:05:54.481112 14131 ts_manager.cc:194] Re-registered known tserver with Master: c49acfe4441c4b0a84aa8e41ac0cc054 (127.7.209.67:37215)
I20260929 03:05:54.481856 14131 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:47017
I20260929 03:05:54.479665 14130 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:05:54.484272 14130 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:54.486377 15373 heartbeater.cc:347] Connected to a master server at 127.7.209.125:44209
I20260929 03:05:54.486491 15373 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:54.486824 15373 heartbeater.cc:511] Master 127.7.209.125:44209 requested a full tablet report, sending...
I20260929 03:05:54.487709 14730 ts_manager.cc:194] Registered new tserver with Master: c49acfe4441c4b0a84aa8e41ac0cc054 (127.7.209.67:37215)
I20260929 03:05:54.488045 15241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: NON_VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: true } } }
I20260929 03:05:54.488197 15241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:54.490710 15385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.462245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34160 (local address 127.7.209.124:35661)
0929 03:05:54.468675 (+ 6430us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.468682 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.468698 (+ 16us) server_negotiation.cc:407] Connection header received
0929 03:05:54.468736 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.468741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.468800 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.468895 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.470207 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.471063 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.475249 (+ 4186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.475987 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.476077 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.477809 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.477813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.477816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.477839 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.488162 (+ 10323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.489080 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.489085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.489087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.489218 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.489585 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.489589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.489590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.489893 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.490039 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.490327 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.490490 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2843,"thread_start_us":103,"threads_started":1}
I20260929 03:05:54.494540 15241 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987: attempting to promote NON_VOTER b399b817a1654b2bac7f203e6edc0243 to VOTER
I20260929 03:05:54.494936 15241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } }
I20260929 03:05:54.496254 14743 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 FOLLOWER]: Refusing update from remote peer 073adfa114bf4d518ffcce398e022987: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260929 03:05:54.496600 14927 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 LEARNER]: Refusing update from remote peer 073adfa114bf4d518ffcce398e022987: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260929 03:05:54.496630 15396 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260929 03:05:54.496937 15241 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260929 03:05:54.498457 15241 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b399b817a1654b2bac7f203e6edc0243 (127.7.209.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } } }
I20260929 03:05:54.498801 14743 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b399b817a1654b2bac7f203e6edc0243 (127.7.209.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } } }
I20260929 03:05:54.499450 14927 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b399b817a1654b2bac7f203e6edc0243 (127.7.209.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } } }
I20260929 03:05:54.501858 14956 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } } }
I20260929 03:05:54.501973 14956 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:05:54.502125 15241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } } }
I20260929 03:05:54.502285 15241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:54.502558 15241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } } }
I20260929 03:05:54.502715 15241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:54.503681 15389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } } }
I20260929 03:05:54.503795 15389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:54.504590 15398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "073adfa114bf4d518ffcce398e022987" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } } }
I20260929 03:05:54.504750 15398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:05:54.507512 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.502609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:34659 (local address 127.0.0.1:40226)
0929 03:05:54.503011 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:54.503027 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:54.503110 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:54.503813 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:54.503816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:54.503836 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:54.503963 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.503969 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.505110 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.505114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.506131 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.506141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.507069 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.507072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.507189 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.507214 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.507276 (+ 62us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.507362 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":1}
I20260929 03:05:54.507849 15403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.502865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40226 (local address 127.7.209.66:34659)
0929 03:05:54.503238 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.503251 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.503279 (+ 28us) server_negotiation.cc:407] Connection header received
0929 03:05:54.503367 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.503374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.503659 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.503964 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:54.504143 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.505000 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.506273 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.506949 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.507058 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.507255 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.507314 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.507431 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":112,"threads_started":1}
I20260929 03:05:54.510242 15401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.502754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:37215 (local address 127.0.0.1:34352)
0929 03:05:54.504040 (+ 1286us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:54.504056 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:54.504166 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:54.505148 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:54.505159 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:54.505177 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:54.505282 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.505288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.507046 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.507049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.508402 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.508413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.509827 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.509830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.509918 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.509943 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.510006 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.510083 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1221,"thread_start_us":95,"threads_started":1}
I20260929 03:05:54.510664 15402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.502788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:39031 (local address 127.0.0.1:40572)
0929 03:05:54.504168 (+ 1380us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:54.504179 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:54.504322 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:54.506193 (+ 1871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:54.506196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:54.506215 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:54.506307 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.506313 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.507984 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.507987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.509120 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.509130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.510278 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.510281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.510373 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.510400 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.510463 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.510536 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1321,"thread_start_us":76,"threads_started":1}
I20260929 03:05:54.510842 15404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.503216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34352 (local address 127.7.209.67:37215)
0929 03:05:54.504496 (+ 1280us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.504524 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.504553 (+ 29us) server_negotiation.cc:407] Connection header received
0929 03:05:54.504642 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.504648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.504906 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.505166 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:54.505584 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.506909 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.508537 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.509471 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.509550 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.509732 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.510415 (+ 683us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.510535 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1150,"thread_start_us":128,"threads_started":1}
I20260929 03:05:54.510911 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.503458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40572 (local address 127.7.209.65:39031)
0929 03:05:54.505488 (+ 2030us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.505529 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.505556 (+ 27us) server_negotiation.cc:407] Connection header received
0929 03:05:54.505674 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.505681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.505890 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.506447 (+ 557us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:54.506609 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.507864 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.509270 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.510155 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.510261 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.510543 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.510589 (+ 46us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.510724 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1910,"thread_start_us":116,"threads_started":1}
I20260929 03:05:54.511821 15167 tablet_service.cc:1467] Processing CreateTablet for tablet 9c1d788fa0474425834ef8a0573c56a8 (DEFAULT_TABLE table=test-table [id=8bdfe49761b74251a9e2ae9df5e29d0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:54.513334 15167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c1d788fa0474425834ef8a0573c56a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:54.514274 15305 tablet_service.cc:1467] Processing CreateTablet for tablet 9c1d788fa0474425834ef8a0573c56a8 (DEFAULT_TABLE table=test-table [id=8bdfe49761b74251a9e2ae9df5e29d0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:54.515470 15305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c1d788fa0474425834ef8a0573c56a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:54.515947 15030 tablet_service.cc:1467] Processing CreateTablet for tablet 9c1d788fa0474425834ef8a0573c56a8 (DEFAULT_TABLE table=test-table [id=8bdfe49761b74251a9e2ae9df5e29d0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:54.516975 15030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c1d788fa0474425834ef8a0573c56a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:54.524438 15406 tablet_bootstrap.cc:492] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347: Bootstrap starting.
I20260929 03:05:54.526022 15407 tablet_bootstrap.cc:492] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697: Bootstrap starting.
I20260929 03:05:54.527511 15406 tablet_bootstrap.cc:654] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:54.528636 15406 log.cc:824] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:54.528687 15407 tablet_bootstrap.cc:654] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:54.530072 15408 tablet_bootstrap.cc:492] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054: Bootstrap starting.
I20260929 03:05:54.530076 15407 log.cc:824] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:54.530830 15406 tablet_bootstrap.cc:492] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347: No bootstrap required, opened a new log
I20260929 03:05:54.531070 15406 ts_tablet_manager.cc:1397] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 03:05:54.532537 15407 tablet_bootstrap.cc:492] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697: No bootstrap required, opened a new log
I20260929 03:05:54.532742 15407 ts_tablet_manager.cc:1397] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 03:05:54.532802 15408 tablet_bootstrap.cc:654] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:54.534085 15408 log.cc:824] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:54.535882 15406 raft_consensus.cc:348] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } } peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } }
I20260929 03:05:54.536201 15406 raft_consensus.cc:374] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:54.536260 15406 raft_consensus.cc:729] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e244ba09d89d4662a64088da114ca347, State: Initialized, Role: FOLLOWER
I20260929 03:05:54.536392 15408 tablet_bootstrap.cc:492] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054: No bootstrap required, opened a new log
I20260929 03:05:54.536582 15408 ts_tablet_manager.cc:1397] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 03:05:54.536939 15406 consensus_queue.cc:260] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } } peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } }
I20260929 03:05:54.538076 15406 ts_tablet_manager.cc:1428] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260929 03:05:54.538353 15097 heartbeater.cc:503] Master 127.7.209.126:43463 was elected leader, sending a full tablet report...
I20260929 03:05:54.538586 15407 raft_consensus.cc:348] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } } peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } }
I20260929 03:05:54.538858 15407 raft_consensus.cc:374] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:54.538909 15407 raft_consensus.cc:729] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b38a7e3aef14c47826af2d89ef37697, State: Initialized, Role: FOLLOWER
I20260929 03:05:54.539377 15407 consensus_queue.cc:260] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } } peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } }
I20260929 03:05:54.540436 15407 ts_tablet_manager.cc:1428] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260929 03:05:54.540798 15234 heartbeater.cc:503] Master 127.7.209.126:43463 was elected leader, sending a full tablet report...
I20260929 03:05:54.543196 15408 raft_consensus.cc:348] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } } peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } }
I20260929 03:05:54.543473 15408 raft_consensus.cc:374] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:54.543524 15408 raft_consensus.cc:729] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c49acfe4441c4b0a84aa8e41ac0cc054, State: Initialized, Role: FOLLOWER
I20260929 03:05:54.544026 15408 consensus_queue.cc:260] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } } peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } }
I20260929 03:05:54.545217 15408 ts_tablet_manager.cc:1428] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260929 03:05:54.545459 15372 heartbeater.cc:503] Master 127.7.209.126:43463 was elected leader, sending a full tablet report...
W20260929 03:05:54.706386 15374 tablet.cc:2369] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:05:54.729277 15236 tablet.cc:2369] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:05:54.741740 15099 tablet.cc:2369] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:05:54.812613 15413 raft_consensus.cc:482] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:54.812920 15413 raft_consensus.cc:504] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } } peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } }
I20260929 03:05:54.814436 15413 leader_election.cc:288] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c49acfe4441c4b0a84aa8e41ac0cc054 (127.7.209.67:37215), e244ba09d89d4662a64088da114ca347 (127.7.209.65:39031)
I20260929 03:05:54.819789 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.814866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:33081 (local address 127.7.209.65:39031)
0929 03:05:54.815100 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.815105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.815180 (+ 75us) server_negotiation.cc:407] Connection header received
0929 03:05:54.815363 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.815366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.815424 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.815505 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:54.815844 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.816723 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.818563 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.819179 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.819305 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.819365 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.819561 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.819642 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260929 03:05:54.819789 15404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.814956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:38521 (local address 127.7.209.67:37215)
0929 03:05:54.815097 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.815101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.815246 (+ 145us) server_negotiation.cc:407] Connection header received
0929 03:05:54.815362 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.815365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.815424 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.815511 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:54.815885 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.816725 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.818563 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.819186 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.819305 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.819365 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.819560 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.819639 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 03:05:54.819818 15416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.814740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:37215 (local address 127.7.209.66:38521)
0929 03:05:54.815166 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:54.815180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:54.815259 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:54.815530 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:54.815533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:54.815564 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:54.815780 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.815787 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.816831 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.816835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.818447 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.818458 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.819298 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.819301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.819433 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.819462 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.819537 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.819642 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":202,"spinlock_wait_cycles":1280,"thread_start_us":111,"threads_started":1}
I20260929 03:05:54.819816 15415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.814740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:39031 (local address 127.7.209.66:33081)
0929 03:05:54.815078 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:54.815096 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:54.815258 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:54.815530 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:54.815533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:54.815578 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:54.815734 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.815744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.816831 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.816835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.818447 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:54.818458 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.819298 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.819301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:54.819430 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.819461 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 03:05:54.819532 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 03:05:54.819638 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":171,"spinlock_wait_cycles":44544,"thread_start_us":97,"threads_started":1}
I20260929 03:05:54.820514 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1d788fa0474425834ef8a0573c56a8" candidate_uuid: "8b38a7e3aef14c47826af2d89ef37697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" is_pre_election: true
I20260929 03:05:54.820727 15050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1d788fa0474425834ef8a0573c56a8" candidate_uuid: "8b38a7e3aef14c47826af2d89ef37697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e244ba09d89d4662a64088da114ca347" is_pre_election: true
I20260929 03:05:54.820880 15325 raft_consensus.cc:2454] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b38a7e3aef14c47826af2d89ef37697 in term 0.
I20260929 03:05:54.821038 15050 raft_consensus.cc:2454] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b38a7e3aef14c47826af2d89ef37697 in term 0.
I20260929 03:05:54.821542 15123 leader_election.cc:302] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b38a7e3aef14c47826af2d89ef37697, c49acfe4441c4b0a84aa8e41ac0cc054; no voters:
I20260929 03:05:54.821842 15413 raft_consensus.cc:2781] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:54.821928 15413 raft_consensus.cc:482] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:54.821959 15413 raft_consensus.cc:3037] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:54.823441 15413 raft_consensus.cc:504] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } } peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } }
I20260929 03:05:54.823789 15413 leader_election.cc:288] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [CANDIDATE]: Term 1 election: Requested vote from peers c49acfe4441c4b0a84aa8e41ac0cc054 (127.7.209.67:37215), e244ba09d89d4662a64088da114ca347 (127.7.209.65:39031)
I20260929 03:05:54.824187 15050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1d788fa0474425834ef8a0573c56a8" candidate_uuid: "8b38a7e3aef14c47826af2d89ef37697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e244ba09d89d4662a64088da114ca347"
I20260929 03:05:54.824158 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c1d788fa0474425834ef8a0573c56a8" candidate_uuid: "8b38a7e3aef14c47826af2d89ef37697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054"
I20260929 03:05:54.824311 15050 raft_consensus.cc:3037] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:54.824303 15325 raft_consensus.cc:3037] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:54.826001 15325 raft_consensus.cc:2454] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b38a7e3aef14c47826af2d89ef37697 in term 1.
I20260929 03:05:54.826004 15050 raft_consensus.cc:2454] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b38a7e3aef14c47826af2d89ef37697 in term 1.
I20260929 03:05:54.826290 15121 leader_election.cc:302] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b38a7e3aef14c47826af2d89ef37697, e244ba09d89d4662a64088da114ca347; no voters:
I20260929 03:05:54.826473 15413 raft_consensus.cc:2781] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:54.826670 15413 raft_consensus.cc:686] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 1 LEADER]: Becoming Leader. State: Replica: 8b38a7e3aef14c47826af2d89ef37697, State: Running, Role: LEADER
I20260929 03:05:54.827121 15413 consensus_queue.cc:237] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } } peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } }
I20260929 03:05:54.831127 14131 catalog_manager.cc:5495] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b38a7e3aef14c47826af2d89ef37697 (127.7.209.66). New cstate: current_term: 1 leader_uuid: "8b38a7e3aef14c47826af2d89ef37697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } health_report { overall_health: UNKNOWN } } }
I20260929 03:05:54.878031 15403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.874096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40230 (local address 127.7.209.66:34659)
0929 03:05:54.874250 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.874255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.874270 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:05:54.874377 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.874379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.874443 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.874541 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:54.875013 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.875564 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.876447 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.876682 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.876770 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.877132 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:54.877651 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:54.877779 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.877816 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.877883 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 03:05:54.886749 15050 raft_consensus.cc:1261] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347 [term 1 FOLLOWER]: Refusing update from remote peer 8b38a7e3aef14c47826af2d89ef37697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:54.886749 15325 raft_consensus.cc:1261] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 1 FOLLOWER]: Refusing update from remote peer 8b38a7e3aef14c47826af2d89ef37697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:54.887698 15413 consensus_queue.cc:1035] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:54.887976 15417 consensus_queue.cc:1035] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:54.899019 15421 mvcc.cc:205] Tried to move back new op lower bound from 7334507130405101568 to 7334507130174832640. Current Snapshot: MvccSnapshot[applied={T|T < 7334507130405101568}]
I20260929 03:05:54.911317 14130 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:54.911528 14130 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:54.916146 14130 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 073adfa114bf4d518ffcce398e022987: Sending DeleteTablet for 3 replicas of tablet 9c1d788fa0474425834ef8a0573c56a8
I20260929 03:05:54.917290 15305 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c1d788fa0474425834ef8a0573c56a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:54 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:34352
I20260929 03:05:54.917865 15167 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c1d788fa0474425834ef8a0573c56a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:54 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:40226
I20260929 03:05:54.918815 15030 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c1d788fa0474425834ef8a0573c56a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:54 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:40572
I20260929 03:05:54.919727 15432 tablet_replica.cc:330] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347: stopping tablet replica
I20260929 03:05:54.920135 15432 raft_consensus.cc:2229] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:54.920521 15432 raft_consensus.cc:2258] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:54.920640 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 14099
I20260929 03:05:54.923885 15432 ts_tablet_manager.cc:1905] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:54.925166 15429 tablet_replica.cc:330] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054: stopping tablet replica
I20260929 03:05:54.925716 15429 raft_consensus.cc:2229] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:54.926188 15429 raft_consensus.cc:2258] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:54.929832 15429 ts_tablet_manager.cc:1905] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:54.930631 15383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.918975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.7.209.125:44209)
0929 03:05:54.919083 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.919087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.919098 (+ 11us) server_negotiation.cc:407] Connection header received
0929 03:05:54.919129 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.919131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.919170 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.919239 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.920569 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.921405 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.922703 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.923041 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.923122 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.926388 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.926393 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.926396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.926427 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.928242 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.928833 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.928837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.928838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.928936 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.929278 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.929281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.929283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.929546 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.929809 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.930308 (+ 499us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.930462 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 03:05:54.932796 15432 ts_tablet_manager.cc:1918] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:54.932910 15432 log.cc:1196] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-0/wal/wals/9c1d788fa0474425834ef8a0573c56a8
I20260929 03:05:54.933285 15432 ts_tablet_manager.cc:1939] T 9c1d788fa0474425834ef8a0573c56a8 P e244ba09d89d4662a64088da114ca347: Deleting consensus metadata
I20260929 03:05:54.934460 15431 tablet_replica.cc:330] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697: stopping tablet replica
I20260929 03:05:54.934829 15431 raft_consensus.cc:2229] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:05:54.935333 15431 raft_consensus.cc:2258] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:54.936288 15429 ts_tablet_manager.cc:1918] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:54.936398 15429 log.cc:1196] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/9c1d788fa0474425834ef8a0573c56a8
I20260929 03:05:54.936759 15429 ts_tablet_manager.cc:1939] T 9c1d788fa0474425834ef8a0573c56a8 P c49acfe4441c4b0a84aa8e41ac0cc054: Deleting consensus metadata
W20260929 03:05:54.938644 14983 connection.cc:441] server connection from 127.0.0.1:40572 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40572 (request call id 1) could send its response
W20260929 03:05:54.939301 15258 connection.cc:441] server connection from 127.0.0.1:34352 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34352 (request call id 1) could send its response
I20260929 03:05:54.939352 15431 ts_tablet_manager.cc:1905] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:54.940828 15385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.918790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34166 (local address 127.7.209.124:35661)
0929 03:05:54.918904 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.918908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.931501 (+ 12593us) server_negotiation.cc:407] Connection header received
0929 03:05:54.931714 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.931718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.931795 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.931853 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.932768 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.933736 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.935067 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.935454 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.935530 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.937402 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.937406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.937408 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.937435 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.939021 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.939649 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.939653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.939655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.939753 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.940002 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.940005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.940007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.940186 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.940306 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.940520 (+ 214us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.940644 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:05:54.942334 15431 ts_tablet_manager.cc:1918] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:54.942435 15431 log.cc:1196] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/9c1d788fa0474425834ef8a0573c56a8
I20260929 03:05:54.942747 15431 ts_tablet_manager.cc:1939] T 9c1d788fa0474425834ef8a0573c56a8 P 8b38a7e3aef14c47826af2d89ef37697: Deleting consensus metadata
W20260929 03:05:54.943817 15122 connection.cc:441] server connection from 127.0.0.1:40226 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40226 (request call id 1) could send its response
I20260929 03:05:54.953122 15383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.944664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44166 (local address 127.7.209.125:44209)
0929 03:05:54.944790 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.944794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.944809 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:05:54.944848 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.944851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.944925 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.944994 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.946409 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.947005 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.948028 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.948364 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.948437 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.950112 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.950115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.950118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.950142 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.951541 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.951973 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.951976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.951978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.952067 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.952332 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.952335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.952337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.952510 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.952627 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.952832 (+ 205us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.952944 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:05:54.954536 15385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:54.945043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34178 (local address 127.7.209.124:35661)
0929 03:05:54.945158 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:54.945162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:54.945405 (+ 243us) server_negotiation.cc:407] Connection header received
0929 03:05:54.945519 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:54.945522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:54.945565 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:54.945674 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:54.946505 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.947310 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.948522 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:54.948880 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:54.948981 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:54.951123 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:54.951127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:54.951130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:54.951154 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:54.952711 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.953390 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.953393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.953395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.953480 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:54.953768 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:54.953771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:54.953773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:54.953991 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:54.954107 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:54.954328 (+ 221us) server_negotiation.cc:299] Negotiation successful
0929 03:05:54.954415 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 03:05:55.902968 15439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:55.901558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.65:44633)
0929 03:05:55.902428 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:55.902574 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":124,"threads_started":1}
I20260929 03:05:55.903424 15440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:55.902437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.67:41291)
0929 03:05:55.902923 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:55.903037 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":97,"threads_started":1}
W20260929 03:05:55.903586 15097 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:43463 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
I20260929 03:05:55.903987 15441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:55.903095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.66:52247)
0929 03:05:55.903766 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:55.903878 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":90,"threads_started":1}
W20260929 03:05:55.904066 15372 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:43463 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
W20260929 03:05:55.904253 15234 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:43463 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
I20260929 03:05:56.416558 15442 raft_consensus.cc:482] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 073adfa114bf4d518ffcce398e022987)
I20260929 03:05:56.416819 15442 raft_consensus.cc:504] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } }
I20260929 03:05:56.417702 15443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 073adfa114bf4d518ffcce398e022987)
I20260929 03:05:56.417932 15443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } }
I20260929 03:05:56.418241 15442 leader_election.cc:288] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 073adfa114bf4d518ffcce398e022987 (127.7.209.126:43463), 1b84e1dc781d456c90c2ceefed504316 (127.7.209.125:44209)
I20260929 03:05:56.418910 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.418461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:41250)
0929 03:05:56.418702 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.418787 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":76,"threads_started":1}
W20260929 03:05:56.419242 14903 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
I20260929 03:05:56.419548 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.419193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:41260)
0929 03:05:56.419377 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.419449 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":73,"threads_started":1}
I20260929 03:05:56.419544 15443 leader_election.cc:288] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 073adfa114bf4d518ffcce398e022987 (127.7.209.126:43463), b399b817a1654b2bac7f203e6edc0243 (127.7.209.124:35661)
W20260929 03:05:56.419775 14720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
I20260929 03:05:56.420297 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.420030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:41266)
0929 03:05:56.420138 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.420206 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 03:05:56.420603 14903 leader_election.cc:334] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 073adfa114bf4d518ffcce398e022987 (127.7.209.126:43463): Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
I20260929 03:05:56.420948 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.420708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:41276)
0929 03:05:56.420822 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.420880 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260929 03:05:56.421089 14720 leader_election.cc:334] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 073adfa114bf4d518ffcce398e022987 (127.7.209.126:43463): Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
I20260929 03:05:56.423448 15447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.419200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35661 (local address 127.0.0.1:34188)
0929 03:05:56.419726 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.419739 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:56.419877 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:56.420298 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:56.420301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:56.420323 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:56.420444 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.420451 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.421429 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.421433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.422371 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.422380 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.423028 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.423032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.423174 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.423195 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 03:05:56.423261 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 03:05:56.423334 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":191,"thread_start_us":64,"threads_started":1}
I20260929 03:05:56.423877 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.418861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:44209 (local address 127.0.0.1:44178)
0929 03:05:56.420133 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.420146 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:56.420245 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:56.420822 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:56.420825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:56.420843 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:56.420936 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.420943 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.421828 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.421831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.422635 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.422643 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.423333 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.423337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.423426 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.423444 (+ 18us) client_negotiation.cc:769] Sending connection context
0929 03:05:56.423680 (+ 236us) client_negotiation.cc:240] Negotiation successful
0929 03:05:56.423756 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260929 03:05:56.423872 15446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.418968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44178 (local address 127.7.209.125:44209)
0929 03:05:56.419264 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:56.419269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:56.420210 (+ 941us) server_negotiation.cc:407] Connection header received
0929 03:05:56.420364 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:56.420367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:56.420414 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:56.420479 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:56.421087 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.421711 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.422750 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.423232 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.423323 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.423378 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:56.423697 (+ 319us) server_negotiation.cc:299] Negotiation successful
0929 03:05:56.423765 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":225,"thread_start_us":74,"threads_started":1}
I20260929 03:05:56.424325 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.419441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34188 (local address 127.7.209.124:35661)
0929 03:05:56.420007 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:56.420016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:56.420033 (+ 17us) server_negotiation.cc:407] Connection header received
0929 03:05:56.420087 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:56.420091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:56.420144 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:56.420282 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:56.420558 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.421330 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.422487 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.422928 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.423018 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.423087 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:56.423283 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 03:05:56.424202 (+ 919us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":499,"thread_start_us":133,"threads_started":1}
I20260929 03:05:56.424558 14743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b399b817a1654b2bac7f203e6edc0243" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1b84e1dc781d456c90c2ceefed504316" is_pre_election: true
I20260929 03:05:56.424742 14743 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b399b817a1654b2bac7f203e6edc0243 in term 1.
I20260929 03:05:56.425086 14903 leader_election.cc:302] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b84e1dc781d456c90c2ceefed504316, b399b817a1654b2bac7f203e6edc0243; no voters: 073adfa114bf4d518ffcce398e022987
I20260929 03:05:56.425334 15442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 03:05:56.425329 14927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b84e1dc781d456c90c2ceefed504316" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b399b817a1654b2bac7f203e6edc0243" is_pre_election: true
I20260929 03:05:56.425416 15442 raft_consensus.cc:482] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 073adfa114bf4d518ffcce398e022987)
I20260929 03:05:56.425455 15442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 1 FOLLOWER]: Advancing to term 2
I20260929 03:05:56.426723 15442 raft_consensus.cc:504] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } }
I20260929 03:05:56.426926 14927 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b84e1dc781d456c90c2ceefed504316 in current term 2: Already voted for candidate b399b817a1654b2bac7f203e6edc0243 in this term.
I20260929 03:05:56.427014 15442 leader_election.cc:288] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [CANDIDATE]: Term 2 election: Requested vote from peers 073adfa114bf4d518ffcce398e022987 (127.7.209.126:43463), 1b84e1dc781d456c90c2ceefed504316 (127.7.209.125:44209)
I20260929 03:05:56.427333 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.427131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:41306)
0929 03:05:56.427228 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.427277 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 03:05:56.427511 14743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b399b817a1654b2bac7f203e6edc0243" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1b84e1dc781d456c90c2ceefed504316"
I20260929 03:05:56.427641 14743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 1 FOLLOWER]: Advancing to term 2
I20260929 03:05:56.427820 14719 leader_election.cc:302] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b84e1dc781d456c90c2ceefed504316; no voters: 073adfa114bf4d518ffcce398e022987, b399b817a1654b2bac7f203e6edc0243
I20260929 03:05:56.427997 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.427790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:41320)
0929 03:05:56.427893 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.427942 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 03:05:56.428159 14903 leader_election.cc:334] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 073adfa114bf4d518ffcce398e022987 (127.7.209.126:43463): Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
I20260929 03:05:56.428815 14743 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b399b817a1654b2bac7f203e6edc0243 in term 2.
I20260929 03:05:56.428925 15443 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 03:05:56.429070 14903 leader_election.cc:302] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b84e1dc781d456c90c2ceefed504316, b399b817a1654b2bac7f203e6edc0243; no voters: 073adfa114bf4d518ffcce398e022987
I20260929 03:05:56.429253 15442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 03:05:56.429441 15442 raft_consensus.cc:686] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [term 2 LEADER]: Becoming Leader. State: Replica: b399b817a1654b2bac7f203e6edc0243, State: Running, Role: LEADER
I20260929 03:05:56.429736 15442 consensus_queue.cc:237] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } }
I20260929 03:05:56.430944 15452 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b399b817a1654b2bac7f203e6edc0243. Latest consensus state: current_term: 2 leader_uuid: "b399b817a1654b2bac7f203e6edc0243" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } } }
I20260929 03:05:56.431039 15452 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:56.431330 15454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:56.433574 15454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:56.434008 15454 catalog_manager.cc:1259] Loaded cluster ID: 8689b40822d54e7ab5d5845ac4e6d649
I20260929 03:05:56.434049 15454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:56.434563 15454 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:56.434931 15454 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: Loaded TSK: 0
I20260929 03:05:56.435724 15454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:05:56.489727 14916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:05:56.494086 14916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:56.498122 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.497784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:41332)
0929 03:05:56.497924 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.498002 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 03:05:56.498149 14743 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [term 2 FOLLOWER]: Refusing update from remote peer b399b817a1654b2bac7f203e6edc0243: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260929 03:05:56.498914 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.498674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.0.0.1:41338)
0929 03:05:56.498782 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.498836 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 03:05:56.499035 14903 consensus_peers.cc:487] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 -> Peer 073adfa114bf4d518ffcce398e022987 (127.7.209.126:43463): Couldn't send request to peer 073adfa114bf4d518ffcce398e022987. Status: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 03:05:56.499048 15452 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260929 03:05:56.500923 15443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b399b817a1654b2bac7f203e6edc0243. Latest consensus state: current_term: 2 leader_uuid: "b399b817a1654b2bac7f203e6edc0243" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } } }
I20260929 03:05:56.501084 15443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:05:56.502332 15442 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b399b817a1654b2bac7f203e6edc0243" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } } }
I20260929 03:05:56.502480 15442 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243 [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:56.503352 15443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b399b817a1654b2bac7f203e6edc0243" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "073adfa114bf4d518ffcce398e022987" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43463 } } peers { permanent_uuid: "1b84e1dc781d456c90c2ceefed504316" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 44209 } attrs { promote: false } } peers { permanent_uuid: "b399b817a1654b2bac7f203e6edc0243" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35661 } attrs { promote: false } } }
I20260929 03:05:56.503475 15443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b84e1dc781d456c90c2ceefed504316 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:05:56.513680 15444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.508784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:34659 (local address 127.0.0.1:40232)
0929 03:05:56.508906 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.508925 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:56.509013 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:56.510158 (+ 1145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:56.510169 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:56.510184 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:56.510295 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.510301 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.511497 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.511500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.512274 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.512283 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.513197 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.513201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.513292 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.513346 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 03:05:56.513403 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 03:05:56.513490 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":59}
I20260929 03:05:56.513682 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.508993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:39031 (local address 127.0.0.1:40574)
0929 03:05:56.509428 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.509442 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:56.509537 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:56.509993 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:56.509996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:56.510013 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:56.510154 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.510162 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.511194 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.511197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.512407 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.512416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.513297 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.513300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.513414 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.513435 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 03:05:56.513472 (+ 37us) client_negotiation.cc:240] Negotiation successful
0929 03:05:56.513534 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":5}
I20260929 03:05:56.514096 15462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.509238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:37215 (local address 127.0.0.1:34366)
0929 03:05:56.509446 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.509460 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:56.509546 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:56.510201 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:56.510204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:56.510222 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:56.510329 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.510335 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.511418 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.511421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.512564 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.512572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.513360 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.513363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.513452 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.513474 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 03:05:56.513539 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 03:05:56.513948 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":30,"thread_start_us":94,"threads_started":1}
I20260929 03:05:56.514240 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.509229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40574 (local address 127.7.209.65:39031)
0929 03:05:56.509769 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:56.509773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:56.509786 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:05:56.509838 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:56.509840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:56.509885 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:56.509969 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:56.510293 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.511082 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.512547 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.513182 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.513864 (+ 682us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.513941 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:56.513983 (+ 42us) server_negotiation.cc:299] Negotiation successful
0929 03:05:56.514081 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":93,"threads_started":1}
I20260929 03:05:56.514359 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.509511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34366 (local address 127.7.209.67:37215)
0929 03:05:56.509825 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:56.509830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:56.509845 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:05:56.509904 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:56.509907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:56.509961 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:56.510061 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:56.510474 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.511313 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.512666 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.513099 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.513221 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.513304 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:56.513682 (+ 378us) server_negotiation.cc:299] Negotiation successful
0929 03:05:56.513775 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":155,"threads_started":1}
I20260929 03:05:56.514714 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.509180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40232 (local address 127.7.209.66:34659)
0929 03:05:56.509888 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:56.509892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:56.509907 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:05:56.509960 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:56.509963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:56.510017 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:56.510137 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:56.510555 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.511411 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.512389 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.513036 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.514147 (+ 1111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.514218 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:56.514254 (+ 36us) server_negotiation.cc:299] Negotiation successful
0929 03:05:56.514337 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":99,"threads_started":1}
I20260929 03:05:56.514752 15030 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1548d2a973475db4283606686f6371 (DEFAULT_TABLE table=test-table [id=5aa09a4900f84011b8603f145957d805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:56.514873 15305 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1548d2a973475db4283606686f6371 (DEFAULT_TABLE table=test-table [id=5aa09a4900f84011b8603f145957d805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:56.515100 15030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1548d2a973475db4283606686f6371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:56.515184 15305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1548d2a973475db4283606686f6371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:56.515225 15167 tablet_service.cc:1467] Processing CreateTablet for tablet 3e1548d2a973475db4283606686f6371 (DEFAULT_TABLE table=test-table [id=5aa09a4900f84011b8603f145957d805]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:56.515477 15167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e1548d2a973475db4283606686f6371. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:56.517983 15466 tablet_bootstrap.cc:492] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054: Bootstrap starting.
I20260929 03:05:56.518478 15467 tablet_bootstrap.cc:492] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697: Bootstrap starting.
I20260929 03:05:56.518827 15466 tablet_bootstrap.cc:654] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:56.519304 15467 tablet_bootstrap.cc:654] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:56.519840 15465 tablet_bootstrap.cc:492] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347: Bootstrap starting.
I20260929 03:05:56.519963 15466 tablet_bootstrap.cc:492] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054: No bootstrap required, opened a new log
I20260929 03:05:56.520057 15466 ts_tablet_manager.cc:1397] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 03:05:56.520380 15467 tablet_bootstrap.cc:492] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697: No bootstrap required, opened a new log
I20260929 03:05:56.520470 15467 ts_tablet_manager.cc:1397] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260929 03:05:56.520567 15466 raft_consensus.cc:348] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } } peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } }
I20260929 03:05:56.520706 15466 raft_consensus.cc:374] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:56.520752 15466 raft_consensus.cc:729] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c49acfe4441c4b0a84aa8e41ac0cc054, State: Initialized, Role: FOLLOWER
I20260929 03:05:56.520891 15466 consensus_queue.cc:260] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } } peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } }
I20260929 03:05:56.520876 15467 raft_consensus.cc:348] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } } peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } }
I20260929 03:05:56.520978 15467 raft_consensus.cc:374] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:56.521008 15467 raft_consensus.cc:729] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b38a7e3aef14c47826af2d89ef37697, State: Initialized, Role: FOLLOWER
I20260929 03:05:56.521209 15466 ts_tablet_manager.cc:1428] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260929 03:05:56.521168 15467 consensus_queue.cc:260] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } } peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } }
I20260929 03:05:56.521912 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.521510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.67:41727)
0929 03:05:56.521748 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.521811 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":107,"threads_started":1}
I20260929 03:05:56.522042 15467 ts_tablet_manager.cc:1428] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 03:05:56.524118 15371 heartbeater.cc:503] Master 127.7.209.124:35661 was elected leader, sending a full tablet report...
I20260929 03:05:56.524206 15465 tablet_bootstrap.cc:654] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:56.524219 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.523715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.67:55387)
0929 03:05:56.524079 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.524136 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284}
I20260929 03:05:56.525878 15465 tablet_bootstrap.cc:492] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347: No bootstrap required, opened a new log
I20260929 03:05:56.525875 15233 heartbeater.cc:503] Master 127.7.209.124:35661 was elected leader, sending a full tablet report...
I20260929 03:05:56.525995 15465 ts_tablet_manager.cc:1397] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260929 03:05:56.526566 15465 raft_consensus.cc:348] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } } peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } }
I20260929 03:05:56.526714 15465 raft_consensus.cc:374] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:56.526752 15465 raft_consensus.cc:729] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e244ba09d89d4662a64088da114ca347, State: Initialized, Role: FOLLOWER
I20260929 03:05:56.526899 15465 consensus_queue.cc:260] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } } peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } }
I20260929 03:05:56.526968 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.522528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.66:57397)
0929 03:05:56.526644 (+ 4116us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.526859 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4033,"thread_start_us":106,"threads_started":1}
I20260929 03:05:56.527273 15465 ts_tablet_manager.cc:1428] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:56.527971 15476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.527544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.66:57627)
0929 03:05:56.527822 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.527882 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":122,"threads_started":1}
I20260929 03:05:56.528676 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.527631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.65:49223)
0929 03:05:56.528521 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.528580 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":112,"threads_started":1}
I20260929 03:05:56.528674 15096 heartbeater.cc:503] Master 127.7.209.124:35661 was elected leader, sending a full tablet report...
I20260929 03:05:56.529431 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.529180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.65:33529)
0929 03:05:56.529271 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.529321 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260929 03:05:56.652527 15470 raft_consensus.cc:482] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:56.652747 15470 raft_consensus.cc:504] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } } peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } }
I20260929 03:05:56.654412 15470 leader_election.cc:288] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b38a7e3aef14c47826af2d89ef37697 (127.7.209.66:34659), e244ba09d89d4662a64088da114ca347 (127.7.209.65:39031)
I20260929 03:05:56.659570 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.655233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:37185 (local address 127.7.209.65:39031)
0929 03:05:56.655374 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:56.655379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:56.655505 (+ 126us) server_negotiation.cc:407] Connection header received
0929 03:05:56.655602 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:56.655604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:56.655653 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:56.655740 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:56.655965 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.656702 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.658307 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.658940 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.659059 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.659120 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:56.659335 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 03:05:56.659428 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 03:05:56.659590 15478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.655054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:39031 (local address 127.7.209.67:37185)
0929 03:05:56.655411 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.655423 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:56.655518 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:56.655754 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:56.655757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:56.655774 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:56.655869 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.655873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.656795 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.656798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.658162 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.658175 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.659103 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.659107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.659207 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.659236 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 03:05:56.659314 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 03:05:56.659392 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":37,"thread_start_us":175,"threads_started":1}
I20260929 03:05:56.659878 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.654840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:36629 (local address 127.7.209.66:34659)
0929 03:05:56.654970 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:56.654974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:56.654993 (+ 19us) server_negotiation.cc:407] Connection header received
0929 03:05:56.655161 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:56.655165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:56.655228 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:56.655324 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:56.655706 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.656549 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.658294 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.658940 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.659060 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.659120 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:56.659652 (+ 532us) server_negotiation.cc:299] Negotiation successful
0929 03:05:56.659759 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 03:05:56.659596 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.654723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:34659 (local address 127.7.209.67:36629)
0929 03:05:56.654885 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.654903 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:56.655036 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:56.655406 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:56.655409 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:56.655432 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:56.655590 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.655597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.656663 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.656666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.658171 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:56.658177 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.659187 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.659192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:56.659296 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.659319 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 03:05:56.659387 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 03:05:56.659460 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":83}
I20260929 03:05:56.660395 15050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1548d2a973475db4283606686f6371" candidate_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e244ba09d89d4662a64088da114ca347" is_pre_election: true
I20260929 03:05:56.660561 15050 raft_consensus.cc:2454] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c49acfe4441c4b0a84aa8e41ac0cc054 in term 0.
I20260929 03:05:56.660576 15187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1548d2a973475db4283606686f6371" candidate_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b38a7e3aef14c47826af2d89ef37697" is_pre_election: true
I20260929 03:05:56.660862 15187 raft_consensus.cc:2454] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c49acfe4441c4b0a84aa8e41ac0cc054 in term 0.
I20260929 03:05:56.660948 15259 leader_election.cc:302] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c49acfe4441c4b0a84aa8e41ac0cc054, e244ba09d89d4662a64088da114ca347; no voters:
I20260929 03:05:56.661233 15470 raft_consensus.cc:2781] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:56.661289 15470 raft_consensus.cc:482] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:56.661315 15470 raft_consensus.cc:3037] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:56.662528 15470 raft_consensus.cc:504] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } } peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } }
I20260929 03:05:56.662856 15470 leader_election.cc:288] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [CANDIDATE]: Term 1 election: Requested vote from peers 8b38a7e3aef14c47826af2d89ef37697 (127.7.209.66:34659), e244ba09d89d4662a64088da114ca347 (127.7.209.65:39031)
I20260929 03:05:56.663198 15187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1548d2a973475db4283606686f6371" candidate_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b38a7e3aef14c47826af2d89ef37697"
I20260929 03:05:56.663327 15187 raft_consensus.cc:3037] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:56.663308 15050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e1548d2a973475db4283606686f6371" candidate_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e244ba09d89d4662a64088da114ca347"
I20260929 03:05:56.663447 15050 raft_consensus.cc:3037] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:56.664391 15187 raft_consensus.cc:2454] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c49acfe4441c4b0a84aa8e41ac0cc054 in term 1.
I20260929 03:05:56.664554 15050 raft_consensus.cc:2454] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c49acfe4441c4b0a84aa8e41ac0cc054 in term 1.
I20260929 03:05:56.664645 15259 leader_election.cc:302] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b38a7e3aef14c47826af2d89ef37697, c49acfe4441c4b0a84aa8e41ac0cc054; no voters:
I20260929 03:05:56.664796 15470 raft_consensus.cc:2781] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:56.665086 15470 raft_consensus.cc:686] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 1 LEADER]: Becoming Leader. State: Replica: c49acfe4441c4b0a84aa8e41ac0cc054, State: Running, Role: LEADER
I20260929 03:05:56.665390 15470 consensus_queue.cc:237] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } } peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } }
I20260929 03:05:56.665795 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.665548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.67:59531)
0929 03:05:56.665683 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.665737 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 03:05:56.668295 14915 catalog_manager.cc:5495] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 reported cstate change: term changed from 0 to 1, leader changed from <none> to c49acfe4441c4b0a84aa8e41ac0cc054 (127.7.209.67). New cstate: current_term: 1 leader_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c49acfe4441c4b0a84aa8e41ac0cc054" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37215 } health_report { overall_health: HEALTHY } } }
I20260929 03:05:56.705684 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.702254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.7.209.67:37215)
0929 03:05:56.702384 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:56.702387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:56.702405 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:05:56.702498 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:56.702500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:56.702563 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:56.702675 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:56.702889 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.703436 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.704280 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:56.704507 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:56.704596 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:56.704779 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:56.705339 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:56.705434 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:56.705462 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 03:05:56.705515 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:05:56.708922 15050 raft_consensus.cc:1261] T 3e1548d2a973475db4283606686f6371 P e244ba09d89d4662a64088da114ca347 [term 1 FOLLOWER]: Refusing update from remote peer c49acfe4441c4b0a84aa8e41ac0cc054: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:56.709275 15187 raft_consensus.cc:1261] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697 [term 1 FOLLOWER]: Refusing update from remote peer c49acfe4441c4b0a84aa8e41ac0cc054: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:56.709460 15470 consensus_queue.cc:1035] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e244ba09d89d4662a64088da114ca347" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 39031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:56.709908 15479 consensus_queue.cc:1035] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b38a7e3aef14c47826af2d89ef37697" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:56.711774 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.711463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.66:35667)
0929 03:05:56.711634 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.711698 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260929 03:05:56.712958 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.711022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.65:39905)
0929 03:05:56.712813 (+ 1791us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.712878 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1712}
I20260929 03:05:56.715301 15473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.715030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.66:41829)
0929 03:05:56.715182 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.715238 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260929 03:05:56.715317 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.715047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.67:55841)
0929 03:05:56.715194 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.715250 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260929 03:05:56.716905 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.716673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.67:55155)
0929 03:05:56.716801 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.716849 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260929 03:05:56.717289 15477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.717076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:43463 (local address 127.7.209.65:52671)
0929 03:05:56.717168 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.717222 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:43463: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260929 03:05:56.723687 14916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:56.723837 14916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:56.727849 14916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b399b817a1654b2bac7f203e6edc0243: Sending DeleteTablet for 3 replicas of tablet 3e1548d2a973475db4283606686f6371
I20260929 03:05:56.728775 15305 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e1548d2a973475db4283606686f6371 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:56 UTC) from {username='oryx', principal='oryx/127.7.209.124@KRBTEST.COM'} at 127.0.0.1:34366
I20260929 03:05:56.728822 15167 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e1548d2a973475db4283606686f6371 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:56 UTC) from {username='oryx', principal='oryx/127.7.209.124@KRBTEST.COM'} at 127.0.0.1:40232
I20260929 03:05:56.729334 15492 tablet_replica.cc:330] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697: stopping tablet replica
I20260929 03:05:56.729521 15492 raft_consensus.cc:2229] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:56.729707 15492 raft_consensus.cc:2258] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:56.729946 15030 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e1548d2a973475db4283606686f6371 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:56 UTC) from {username='oryx', principal='oryx/127.7.209.124@KRBTEST.COM'} at 127.0.0.1:40574
I20260929 03:05:56.729991 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 14879
I20260929 03:05:56.731281 15491 tablet_replica.cc:330] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054: stopping tablet replica
I20260929 03:05:56.731460 15491 raft_consensus.cc:2229] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:05:56.731912 15491 raft_consensus.cc:2258] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:56.732129 15492 ts_tablet_manager.cc:1905] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:56.732735 15491 ts_tablet_manager.cc:1905] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:56.734679 15492 ts_tablet_manager.cc:1918] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:56.734752 15492 log.cc:1196] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/3e1548d2a973475db4283606686f6371
I20260929 03:05:56.735023 15492 ts_tablet_manager.cc:1939] T 3e1548d2a973475db4283606686f6371 P 8b38a7e3aef14c47826af2d89ef37697: Deleting consensus metadata
I20260929 03:05:56.735496 15491 ts_tablet_manager.cc:1918] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:56.735565 15491 log.cc:1196] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/3e1548d2a973475db4283606686f6371
I20260929 03:05:56.735741 14902 catalog_manager.cc:4841] TS 8b38a7e3aef14c47826af2d89ef37697 (127.7.209.66:34659): tablet 3e1548d2a973475db4283606686f6371 (table test-table [id=5aa09a4900f84011b8603f145957d805]) successfully deleted
I20260929 03:05:56.735812 15491 ts_tablet_manager.cc:1939] T 3e1548d2a973475db4283606686f6371 P c49acfe4441c4b0a84aa8e41ac0cc054: Deleting consensus metadata
I20260929 03:05:56.736475 14904 catalog_manager.cc:4841] TS c49acfe4441c4b0a84aa8e41ac0cc054 (127.7.209.67:37215): tablet 3e1548d2a973475db4283606686f6371 (table test-table [id=5aa09a4900f84011b8603f145957d805]) successfully deleted
I20260929 03:05:56.738390 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 15104
I20260929 03:05:56.740218 15462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:56.739950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:39031 (local address 127.0.0.1:40576)
0929 03:05:56.740080 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:56.740135 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 03:05:56.740408 14902 catalog_manager.cc:4584] TS e244ba09d89d4662a64088da114ca347 (127.7.209.65:39031): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e1548d2a973475db4283606686f6371: Network error: Client connection negotiation failed: client connection to 127.7.209.65:39031: connect: Connection refused (error 111)
I20260929 03:05:56.746836 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 15240
I20260929 03:05:56.751992 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 14595
I20260929 03:05:56.756811 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 14706
2026-09-29T03:05:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5825 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:05:56 dist-test-slave-bdps krb5kdc[15495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:05:56 dist-test-slave-bdps krb5kdc[15495](info): set up 2 sockets
Sep 29 03:05:56 dist-test-slave-bdps krb5kdc[15495](info): commencing operation
krb5kdc: starting...
W20260929 03:05:57.714542 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:05:57 dist-test-slave-bdps krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:05:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:05:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.126.keytab.
Entry for principal oryx/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.126.keytab.
I20260929 03:05:57.842466 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:45229
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.126.keytab
--principal=oryx/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:46293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:45229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:57.969125 15508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:57.969504 15508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:57.969570 15508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:57.973975 15508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:05:57.974045 15508 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 03:05:57.974067 15508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:57.974092 15508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:57.974123 15508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:05:57.974152 15508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:05:57.979882 15508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:46293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:45229
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.126.keytab
--principal=oryx/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:45229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:57.981184 15508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:57.982450 15508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:57.991673 15516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:57.991703 15514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:57.991698 15508 server_base.cc:1034] running on GCE node
W20260929 03:05:57.991719 15513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:57.992440 15508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:57.993670 15508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:57.994846 15508 hybrid_clock.cc:648] HybridClock initialized: now 1790651157994829 us; error 32 us; skew 500 ppm
Sep 29 03:05:57 dist-test-slave-bdps krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651157, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:57.997906 15508 init.cc:377] Logged in from keytab as oryx/127.7.209.126@KRBTEST.COM (short username oryx)
I20260929 03:05:57.998911 15508 webserver.cc:460] Webserver started at http://127.7.209.126:36469/ using document root <none> and password file <none>
I20260929 03:05:57.999526 15508 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:57.999584 15508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:57.999807 15508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:58.002100 15508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "2ec35d50a35144cc8c0dc32690c4dd1b"
format_stamp: "Formatted at 2026-09-29 03:05:57 on dist-test-slave-bdps"
server_key: "16f6767d09b1775915447dc752b5b57a"
server_key_iv: "2d02453a839dae8dd0a3650e8f40a1d5"
server_key_version: "encryptionkey@0"
I20260929 03:05:58.002694 15508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "2ec35d50a35144cc8c0dc32690c4dd1b"
format_stamp: "Formatted at 2026-09-29 03:05:57 on dist-test-slave-bdps"
server_key: "16f6767d09b1775915447dc752b5b57a"
server_key_iv: "2d02453a839dae8dd0a3650e8f40a1d5"
server_key_version: "encryptionkey@0"
I20260929 03:05:58.006949 15508 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260929 03:05:58.009847 15523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:58.011287 15508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:58.011412 15508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "2ec35d50a35144cc8c0dc32690c4dd1b"
format_stamp: "Formatted at 2026-09-29 03:05:57 on dist-test-slave-bdps"
server_key: "16f6767d09b1775915447dc752b5b57a"
server_key_iv: "2d02453a839dae8dd0a3650e8f40a1d5"
server_key_version: "encryptionkey@0"
I20260929 03:05:58.011528 15508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:58.022310 15508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:58.025468 15508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:58.025674 15508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:58.034358 15508 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:45229
I20260929 03:05:58.034364 15585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:45229 every 8 connection(s)
I20260929 03:05:58.035408 15508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:05:58.038753 15586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:58.039644 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 15508
I20260929 03:05:58.039795 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:05:58.040129 8005 external_mini_cluster.cc:1428] Setting key 3cdc5c57239b5d733f6e57ed789f9f50
I20260929 03:05:58.046666 15586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b: Bootstrap starting.
I20260929 03:05:58.049927 15586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b: Neither blocks nor log segments found. Creating new log.
Sep 29 03:05:58 dist-test-slave-bdps krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:58.051018 15586 log.cc:824] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:58.053992 15586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b: No bootstrap required, opened a new log
I20260929 03:05:58.058102 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.041669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56784 (local address 127.7.209.126:45229)
0929 03:05:58.042128 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.042151 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.042204 (+ 53us) server_negotiation.cc:407] Connection header received
0929 03:05:58.043093 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.043121 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.043622 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.044070 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:58.044863 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.046512 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.047480 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.048262 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.048446 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.051531 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:58.051537 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:58.051562 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:58.051594 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:58.054477 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.054967 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.054973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.054981 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.055137 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.055377 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.055381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.055383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.055971 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:58.056178 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.056420 (+ 242us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.056805 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":115,"threads_started":1}
I20260929 03:05:58.058614 15586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec35d50a35144cc8c0dc32690c4dd1b" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45229 } }
I20260929 03:05:58.058893 15586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:58.058954 15586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec35d50a35144cc8c0dc32690c4dd1b, State: Initialized, Role: FOLLOWER
I20260929 03:05:58.059485 15586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec35d50a35144cc8c0dc32690c4dd1b" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45229 } }
I20260929 03:05:58.059640 15586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:05:58.059710 15586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:05:58.059821 15586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:58.060920 15586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec35d50a35144cc8c0dc32690c4dd1b" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45229 } }
I20260929 03:05:58.061334 15586 leader_election.cc:302] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ec35d50a35144cc8c0dc32690c4dd1b; no voters:
I20260929 03:05:58.061750 15586 leader_election.cc:288] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:05:58.061904 15591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:58.062147 15591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [term 1 LEADER]: Becoming Leader. State: Replica: 2ec35d50a35144cc8c0dc32690c4dd1b, State: Running, Role: LEADER
I20260929 03:05:58.062538 15591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec35d50a35144cc8c0dc32690c4dd1b" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45229 } }
I20260929 03:05:58.063421 15586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:05:58.064147 15592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ec35d50a35144cc8c0dc32690c4dd1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec35d50a35144cc8c0dc32690c4dd1b" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45229 } } }
I20260929 03:05:58.064292 15592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:58.064692 15593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ec35d50a35144cc8c0dc32690c4dd1b. Latest consensus state: current_term: 1 leader_uuid: "2ec35d50a35144cc8c0dc32690c4dd1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec35d50a35144cc8c0dc32690c4dd1b" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45229 } } }
I20260929 03:05:58.064797 15593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b [sys.catalog]: This master's current role is: LEADER
I20260929 03:05:58.065085 15597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:05:58.068935 15597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:05:58.075567 15597 catalog_manager.cc:1347] Generated new cluster ID: f286469503da45a5838986a08877ccb5
I20260929 03:05:58.075659 15597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:05:58.093474 15597 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:05:58.094719 15597 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:05:58.103410 15597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b: Generated new TSK 0
I20260929 03:05:58.104344 15597 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab.
Entry for principal oryx/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab.
I20260929 03:05:58.140480 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:45229
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab
--principal=oryx/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:46293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:58.292555 15612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:58.292877 15612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:58.292925 15612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:58.297004 15612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:58.297073 15612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:58.297125 15612 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 03:05:58.297178 15612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:05:58.302542 15612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:46293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab
--principal=oryx/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.209.126:45229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:58.303776 15612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:58.304965 15612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:58.315181 15620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:58.315191 15618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:58.315177 15617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:58.315498 15612 server_base.cc:1034] running on GCE node
I20260929 03:05:58.315975 15612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:58.316609 15612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:58.317800 15612 hybrid_clock.cc:648] HybridClock initialized: now 1790651158317780 us; error 37 us; skew 500 ppm
Sep 29 03:05:58 dist-test-slave-bdps krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651158, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:58.320806 15612 init.cc:377] Logged in from keytab as oryx/127.7.209.65@KRBTEST.COM (short username oryx)
I20260929 03:05:58.321710 15612 webserver.cc:460] Webserver started at http://127.7.209.65:41323/ using document root <none> and password file <none>
I20260929 03:05:58.322283 15612 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:58.322347 15612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:58.322564 15612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:58.324820 15612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "698950cba28047819f5cb12812d1ae2a"
format_stamp: "Formatted at 2026-09-29 03:05:58 on dist-test-slave-bdps"
server_key: "67a755e0769e422fefb31a27c19252ef"
server_key_iv: "33c337c95d691a75cb8fff8874cbe6d7"
server_key_version: "encryptionkey@0"
I20260929 03:05:58.325431 15612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "698950cba28047819f5cb12812d1ae2a"
format_stamp: "Formatted at 2026-09-29 03:05:58 on dist-test-slave-bdps"
server_key: "67a755e0769e422fefb31a27c19252ef"
server_key_iv: "33c337c95d691a75cb8fff8874cbe6d7"
server_key_version: "encryptionkey@0"
I20260929 03:05:58.329833 15612 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:05:58.332607 15627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:58.334071 15612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:58.334187 15612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "698950cba28047819f5cb12812d1ae2a"
format_stamp: "Formatted at 2026-09-29 03:05:58 on dist-test-slave-bdps"
server_key: "67a755e0769e422fefb31a27c19252ef"
server_key_iv: "33c337c95d691a75cb8fff8874cbe6d7"
server_key_version: "encryptionkey@0"
I20260929 03:05:58.334291 15612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:58.346994 15612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:58.350121 15612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:58.350311 15612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:58.351977 15612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:58.352028 15612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:58.352085 15612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:58.352114 15612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 03:05:58 dist-test-slave-bdps krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651158, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:58.363719 15612 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:37335
I20260929 03:05:58.363752 15742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:37335 every 8 connection(s)
I20260929 03:05:58.365212 15612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:05:58.367887 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.352917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:38017 (local address 127.7.209.126:45229)
0929 03:05:58.353051 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.353055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.354276 (+ 1221us) server_negotiation.cc:407] Connection header received
0929 03:05:58.355459 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.355463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.355537 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.355623 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:58.357452 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.358207 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.359454 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.359866 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.359999 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.363555 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:58.363559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:58.363563 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:58.363606 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:58.365378 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.366056 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.366060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.366063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.366190 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.366628 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.366632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.366634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.366913 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:58.367003 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.367551 (+ 548us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.367716 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:05:58.368948 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.353536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:45229 (local address 127.7.209.65:38017)
0929 03:05:58.354108 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:58.354155 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:58.355197 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:58.355897 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:58.355909 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:58.356447 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:58.357266 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.357289 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.358339 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.358343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.359309 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.359320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.360080 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.360085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.360302 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.361548 (+ 1246us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:58.361566 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:58.363278 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:58.365506 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:58.365512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:58.365528 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:58.365917 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:58.366295 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:58.366299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:58.366301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:58.366509 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:58.367007 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:58.367013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:58.367320 (+ 307us) client_negotiation.cc:769] Sending connection context
0929 03:05:58.367497 (+ 177us) client_negotiation.cc:240] Negotiation successful
0929 03:05:58.367880 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":148,"threads_started":1}
I20260929 03:05:58.370517 15743 heartbeater.cc:347] Connected to a master server at 127.7.209.126:45229
I20260929 03:05:58.370685 15743 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:58.371325 15743 heartbeater.cc:511] Master 127.7.209.126:45229 requested a full tablet report, sending...
I20260929 03:05:58.371429 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 15612
I20260929 03:05:58.371587 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:05:58.372001 8005 external_mini_cluster.cc:1428] Setting key 4d8d7fca5cb46805c599300debb878c5
I20260929 03:05:58.373335 15540 ts_manager.cc:194] Registered new tserver with Master: 698950cba28047819f5cb12812d1ae2a (127.7.209.65:37335)
I20260929 03:05:58.374962 15540 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:38017
I20260929 03:05:58.381129 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.373585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:59791 (local address 127.7.209.126:45229)
0929 03:05:58.373738 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.373743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.373756 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:05:58.373793 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.373796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.373837 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.373911 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:58.374555 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.375281 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.376444 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.376787 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.376862 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.378270 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:58.378274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:58.378277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:58.378300 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:58.379583 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.379969 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.379972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.379975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.380054 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.380370 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.380372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.380373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.380545 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:58.380620 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.380899 (+ 279us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.381003 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 03:05:58.381228 15749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.373169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:45229 (local address 127.7.209.65:59791)
0929 03:05:58.373506 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:58.373523 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:58.373644 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:58.374036 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:58.374039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:58.374287 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:58.374423 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.374431 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.375437 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.375441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.376307 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.376319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.377020 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.377023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.377118 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.377759 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:58.377770 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:58.378132 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:58.379686 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:58.379689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:58.379692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:58.379884 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:58.380165 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:58.380168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:58.380171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:58.380282 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:58.380628 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:58.380631 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:58.380743 (+ 112us) client_negotiation.cc:769] Sending connection context
0929 03:05:58.380863 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 03:05:58.381018 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":1,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for oryx/127.7.209.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab.
Entry for principal oryx/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab.
I20260929 03:05:58.400259 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:45229
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab
--principal=oryx/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:46293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:58.527688 15752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:58.528023 15752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:58.528108 15752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:58.532305 15752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:58.532373 15752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:58.532420 15752 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 03:05:58.532478 15752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:05:58.538230 15752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:46293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab
--principal=oryx/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.209.126:45229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:58.539506 15752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:58.540829 15752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:58.551301 15757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:58.551301 15758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:58.551301 15760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:58.551631 15752 server_base.cc:1034] running on GCE node
I20260929 03:05:58.552040 15752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:58.552716 15752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:58.553869 15752 hybrid_clock.cc:648] HybridClock initialized: now 1790651158553848 us; error 35 us; skew 500 ppm
Sep 29 03:05:58 dist-test-slave-bdps krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651158, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:58.556795 15752 init.cc:377] Logged in from keytab as oryx/127.7.209.66@KRBTEST.COM (short username oryx)
I20260929 03:05:58.557837 15752 webserver.cc:460] Webserver started at http://127.7.209.66:42713/ using document root <none> and password file <none>
I20260929 03:05:58.558436 15752 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:58.558499 15752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:58.558737 15752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:58.560978 15752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "caa1c916bdf44b60bb50948f340c83f7"
format_stamp: "Formatted at 2026-09-29 03:05:58 on dist-test-slave-bdps"
server_key: "2f76267e062abc603a7bebfd4d4fb37a"
server_key_iv: "02be021815d580fe0cc7a876087c2d0f"
server_key_version: "encryptionkey@0"
I20260929 03:05:58.561548 15752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "caa1c916bdf44b60bb50948f340c83f7"
format_stamp: "Formatted at 2026-09-29 03:05:58 on dist-test-slave-bdps"
server_key: "2f76267e062abc603a7bebfd4d4fb37a"
server_key_iv: "02be021815d580fe0cc7a876087c2d0f"
server_key_version: "encryptionkey@0"
I20260929 03:05:58.565941 15752 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:05:58.568822 15767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:58.570348 15752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:58.570472 15752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "caa1c916bdf44b60bb50948f340c83f7"
format_stamp: "Formatted at 2026-09-29 03:05:58 on dist-test-slave-bdps"
server_key: "2f76267e062abc603a7bebfd4d4fb37a"
server_key_iv: "02be021815d580fe0cc7a876087c2d0f"
server_key_version: "encryptionkey@0"
I20260929 03:05:58.570611 15752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:58.595324 15752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:58.598296 15752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:58.598448 15752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:58.600015 15752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:58.600067 15752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:58.600107 15752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:58.600134 15752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 03:05:58 dist-test-slave-bdps krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651158, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:58.612061 15752 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:33213
I20260929 03:05:58.612707 15882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:33213 every 8 connection(s)
I20260929 03:05:58.613307 15752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:05:58.614791 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.600940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:47589 (local address 127.7.209.126:45229)
0929 03:05:58.601067 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.601071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.602235 (+ 1164us) server_negotiation.cc:407] Connection header received
0929 03:05:58.603402 (+ 1167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.603406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.603492 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.603581 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:58.604999 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.605811 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.607129 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.607378 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.607473 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.610805 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:58.610809 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:58.610812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:58.610845 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:58.612413 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.613032 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.613035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.613037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.613125 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.613491 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.613495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.613496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.613741 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:58.613842 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.614443 (+ 601us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.614582 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 03:05:58.615746 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.601539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:45229 (local address 127.7.209.66:47589)
0929 03:05:58.602074 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:58.602118 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:58.603154 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:58.603827 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:58.603837 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:58.604251 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:58.604847 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.604863 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.605936 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.605940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.607008 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.607018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.607547 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.607551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.607792 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.608495 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:58.608524 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:58.610619 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:58.612518 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:58.612524 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:58.612543 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:58.612940 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:58.613205 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:58.613209 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:58.613211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:58.613350 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:58.613893 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:58.613899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:58.614205 (+ 306us) client_negotiation.cc:769] Sending connection context
0929 03:05:58.614384 (+ 179us) client_negotiation.cc:240] Negotiation successful
0929 03:05:58.614740 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":138,"threads_started":1}
I20260929 03:05:58.617120 15883 heartbeater.cc:347] Connected to a master server at 127.7.209.126:45229
I20260929 03:05:58.617277 15883 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:58.617934 15883 heartbeater.cc:511] Master 127.7.209.126:45229 requested a full tablet report, sending...
I20260929 03:05:58.618000 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 15752
I20260929 03:05:58.618139 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:05:58.618474 8005 external_mini_cluster.cc:1428] Setting key 055c0c542c00964a1051c1d767659950
I20260929 03:05:58.619175 15539 ts_manager.cc:194] Registered new tserver with Master: caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213)
I20260929 03:05:58.619954 15539 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:47589
I20260929 03:05:58.627565 15889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.619690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:45229 (local address 127.7.209.66:52163)
0929 03:05:58.620428 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:58.620443 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:58.620566 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:58.620836 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:58.620840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:58.621048 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:58.621183 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.621191 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.622219 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.622222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.622802 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.622810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.623324 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.623326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.623396 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.623875 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:58.623888 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:58.624186 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:58.626059 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:58.626061 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:58.626063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:58.626256 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:58.626537 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:58.626540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:58.626541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:58.626617 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:58.627133 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:58.627136 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:58.627229 (+ 93us) client_negotiation.cc:769] Sending connection context
0929 03:05:58.627320 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 03:05:58.627433 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":98,"threads_started":1}
I20260929 03:05:58.627681 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.620008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:52163 (local address 127.7.209.126:45229)
0929 03:05:58.620219 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.620224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.620506 (+ 282us) server_negotiation.cc:407] Connection header received
0929 03:05:58.620665 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.620668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.620718 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.620801 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:58.621297 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.622049 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.622911 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.623226 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.623329 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.624311 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:58.624314 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:58.624317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:58.624339 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:58.625952 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.626348 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.626352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.626353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.626439 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.626698 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.626702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.626703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.627008 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:58.627128 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.627389 (+ 261us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.627514 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
WARNING: no policy specified for oryx/127.7.209.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab.
Entry for principal oryx/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab.
I20260929 03:05:58.649065 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:45229
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab
--principal=oryx/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:46293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:05:58.774871 15892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:05:58.775205 15892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:05:58.775256 15892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:05:58.779520 15892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:05:58.779587 15892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:05:58.779627 15892 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 03:05:58.779670 15892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:05:58.785149 15892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:46293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab
--principal=oryx/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.209.126:45229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:05:58.786429 15892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:05:58.787688 15892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:05:58.798604 15900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:58.798611 15898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:05:58.798643 15897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:05:58.798823 15892 server_base.cc:1034] running on GCE node
I20260929 03:05:58.799364 15892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:05:58.800071 15892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:05:58.801214 15892 hybrid_clock.cc:648] HybridClock initialized: now 1790651158801185 us; error 45 us; skew 500 ppm
Sep 29 03:05:58 dist-test-slave-bdps krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651158, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:05:58.804188 15892 init.cc:377] Logged in from keytab as oryx/127.7.209.67@KRBTEST.COM (short username oryx)
I20260929 03:05:58.805189 15892 webserver.cc:460] Webserver started at http://127.7.209.67:33421/ using document root <none> and password file <none>
I20260929 03:05:58.805915 15892 fs_manager.cc:359] Metadata directory not provided
I20260929 03:05:58.805991 15892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:05:58.806249 15892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:05:58.809005 15892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "57f74244291443d8a0c9fd12598681e9"
format_stamp: "Formatted at 2026-09-29 03:05:58 on dist-test-slave-bdps"
server_key: "57cc5fa50d6efab7c649c5780220fea7"
server_key_iv: "06fa9b63088a26b714b66bc339221c41"
server_key_version: "encryptionkey@0"
I20260929 03:05:58.809813 15892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "57f74244291443d8a0c9fd12598681e9"
format_stamp: "Formatted at 2026-09-29 03:05:58 on dist-test-slave-bdps"
server_key: "57cc5fa50d6efab7c649c5780220fea7"
server_key_iv: "06fa9b63088a26b714b66bc339221c41"
server_key_version: "encryptionkey@0"
I20260929 03:05:58.814235 15892 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:05:58.816995 15907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:58.818419 15892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:05:58.818531 15892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "57f74244291443d8a0c9fd12598681e9"
format_stamp: "Formatted at 2026-09-29 03:05:58 on dist-test-slave-bdps"
server_key: "57cc5fa50d6efab7c649c5780220fea7"
server_key_iv: "06fa9b63088a26b714b66bc339221c41"
server_key_version: "encryptionkey@0"
I20260929 03:05:58.818629 15892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:05:58.843473 15892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:05:58.846961 15892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:05:58.847187 15892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:05:58.849052 15892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:05:58.849110 15892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:05:58.849170 15892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:05:58.849198 15892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 03:05:58 dist-test-slave-bdps krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651158, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:58.861043 15892 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:35125
I20260929 03:05:58.861749 16022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:35125 every 8 connection(s)
I20260929 03:05:58.862126 15892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:05:58.863696 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.850004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:60559 (local address 127.7.209.126:45229)
0929 03:05:58.850194 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.850200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.851280 (+ 1080us) server_negotiation.cc:407] Connection header received
0929 03:05:58.852746 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.852751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.852828 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.852922 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:58.854396 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.855018 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.856351 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.856608 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.856719 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.859942 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:58.859946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:58.859949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:58.859982 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:58.861546 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.862085 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.862089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.862091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.862189 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.862500 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.862504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.862506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.862765 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:58.862869 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.863434 (+ 565us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.863551 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 03:05:58.864619 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.850497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:45229 (local address 127.7.209.67:60559)
0929 03:05:58.851064 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:58.851132 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:58.852408 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:58.853156 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:58.853166 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:58.853580 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:58.854222 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.854237 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.855146 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.855149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.856223 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.856234 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.856795 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.856799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.856989 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.857700 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:58.857724 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:58.859778 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:58.861665 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:58.861672 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:58.861687 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:58.861992 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:58.862268 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:58.862271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:58.862273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:58.862414 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:58.862879 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:58.862887 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:58.863185 (+ 298us) client_negotiation.cc:769] Sending connection context
0929 03:05:58.863379 (+ 194us) client_negotiation.cc:240] Negotiation successful
0929 03:05:58.863663 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":117,"threads_started":1}
I20260929 03:05:58.865923 16023 heartbeater.cc:347] Connected to a master server at 127.7.209.126:45229
I20260929 03:05:58.866053 16023 heartbeater.cc:464] Registering TS with master...
I20260929 03:05:58.866566 16023 heartbeater.cc:511] Master 127.7.209.126:45229 requested a full tablet report, sending...
I20260929 03:05:58.867475 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 15892
I20260929 03:05:58.867630 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:05:58.867697 15540 ts_manager.cc:194] Registered new tserver with Master: 57f74244291443d8a0c9fd12598681e9 (127.7.209.67:35125)
I20260929 03:05:58.867975 8005 external_mini_cluster.cc:1428] Setting key 7de6758f2744d09dec63ef52280ad48d
I20260929 03:05:58.868480 15540 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:60559
I20260929 03:05:58.870146 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:05:58.875367 16029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.868233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:45229 (local address 127.7.209.67:49241)
0929 03:05:58.868560 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:58.868574 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:58.868683 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:58.868994 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:58.868997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:58.869198 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:58.869316 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.869323 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.870353 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.870356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.871160 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.871169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.871717 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.871719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.871789 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.872192 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:58.872202 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:58.872474 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:58.874036 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:58.874040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:58.874041 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:58.874238 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:58.874487 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:58.874490 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:58.874492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:58.874569 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:58.874889 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:58.874892 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:58.874975 (+ 83us) client_negotiation.cc:769] Sending connection context
0929 03:05:58.875059 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 03:05:58.875195 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":100,"threads_started":1}
I20260929 03:05:58.875747 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.868303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:49241 (local address 127.7.209.126:45229)
0929 03:05:58.868405 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.868409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.868636 (+ 227us) server_negotiation.cc:407] Connection header received
0929 03:05:58.868765 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.868769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.868822 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.868890 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:58.869443 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.870199 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.871277 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.871615 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.871708 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.872595 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:58.872598 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:58.872601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:58.872626 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:58.873936 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.874326 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.874329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.874331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.874409 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.874644 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.874647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.874649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.874808 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:58.874890 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.875183 (+ 293us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.875292 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
Sep 29 03:05:58 dist-test-slave-bdps krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651158, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 03:05:58 dist-test-slave-bdps krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:05:58 dist-test-slave-bdps krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 03:05:58.896059 15589 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:05:58.889539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56796 (local address 127.7.209.126:45229)
0929 03:05:58.889714 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.889718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.889730 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:05:58.889774 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.889777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.889818 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.889889 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:58.890676 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.891249 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.892169 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.892403 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.892497 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.895908 (+ 3411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56796: BlockingRecv error: recv got EOF from 127.0.0.1:56796 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 29 03:05:58 dist-test-slave-bdps krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651158, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:58.904706 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.897474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56798 (local address 127.7.209.126:45229)
0929 03:05:58.897635 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.897639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.897779 (+ 140us) server_negotiation.cc:407] Connection header received
0929 03:05:58.897875 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.897877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.897918 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.897983 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:58.898524 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.899030 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.899775 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.899983 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.900050 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.902178 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:58.902182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:58.902184 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:58.902211 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:58.903420 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.903789 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.903791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.903793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.903864 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:58.904047 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:58.904049 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:58.904050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:58.904215 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:58.904313 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.904472 (+ 159us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.904566 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 03:05:58.907737 15539 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:05:58.911319 15539 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:58.927778 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.921809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39724 (local address 127.7.209.67:35125)
0929 03:05:58.922270 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.922281 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.922303 (+ 22us) server_negotiation.cc:407] Connection header received
0929 03:05:58.922421 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.922426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.922606 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.922744 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:58.923171 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.924262 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.926214 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.926946 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.927058 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.927251 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.927441 (+ 190us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.927562 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":115,"threads_started":1}
I20260929 03:05:58.927767 16047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.921569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:35125 (local address 127.0.0.1:39724)
0929 03:05:58.922215 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:58.922227 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:58.922314 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:58.922898 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:58.922901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:58.922919 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:58.923052 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.923058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.924386 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.924389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.926064 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.926075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.927075 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.927079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.927199 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.927271 (+ 72us) client_negotiation.cc:769] Sending connection context
0929 03:05:58.927388 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 03:05:58.927565 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":265,"thread_start_us":117,"threads_started":1}
I20260929 03:05:58.927767 16045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.920782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:33213 (local address 127.0.0.1:41904)
0929 03:05:58.921145 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:58.921218 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:58.921357 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:58.922728 (+ 1371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:58.922731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:58.922744 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:58.922866 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.922873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.924316 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.924319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.926064 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.926075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.927130 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.927138 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.927215 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.927268 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 03:05:58.927377 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 03:05:58.927562 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":211,"thread_start_us":81,"threads_started":1}
I20260929 03:05:58.928267 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.921064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:37335 (local address 127.0.0.1:54418)
0929 03:05:58.921900 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:58.921914 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:58.922005 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:58.922613 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:58.922619 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:58.922648 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:58.923050 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.923058 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.924726 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.924729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.925645 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.925657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.926629 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.926633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.926756 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.926803 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 03:05:58.927687 (+ 884us) client_negotiation.cc:240] Negotiation successful
0929 03:05:58.927772 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":61,"thread_start_us":107,"threads_started":1}
I20260929 03:05:58.928391 16049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.921712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54418 (local address 127.7.209.65:37335)
0929 03:05:58.922026 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.922034 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.922060 (+ 26us) server_negotiation.cc:407] Connection header received
0929 03:05:58.922150 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.922156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.922448 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.922622 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:58.923181 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.924619 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.925752 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.926504 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.926618 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.926794 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.928035 (+ 1241us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.928159 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":85,"threads_started":1}
I20260929 03:05:58.928869 16046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.920955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41904 (local address 127.7.209.66:33213)
0929 03:05:58.922263 (+ 1308us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.922276 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.922301 (+ 25us) server_negotiation.cc:407] Connection header received
0929 03:05:58.922384 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.922389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.922564 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.922836 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:58.922986 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.924214 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.926205 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.927003 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.927817 (+ 814us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.928560 (+ 743us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.928626 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.928720 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1195,"thread_start_us":135,"threads_started":1}
I20260929 03:05:58.931813 15677 tablet_service.cc:1467] Processing CreateTablet for tablet 4e81163afdb44f1eb80e836b56e455a0 (DEFAULT_TABLE table=test-table [id=4644ef8da8a142f58472e977a3f4e4fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:58.931825 15957 tablet_service.cc:1467] Processing CreateTablet for tablet 4e81163afdb44f1eb80e836b56e455a0 (DEFAULT_TABLE table=test-table [id=4644ef8da8a142f58472e977a3f4e4fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:58.931823 15817 tablet_service.cc:1467] Processing CreateTablet for tablet 4e81163afdb44f1eb80e836b56e455a0 (DEFAULT_TABLE table=test-table [id=4644ef8da8a142f58472e977a3f4e4fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:58.932936 15677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e81163afdb44f1eb80e836b56e455a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:58.932938 15957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e81163afdb44f1eb80e836b56e455a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:58.932938 15817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e81163afdb44f1eb80e836b56e455a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:58.940079 16051 tablet_bootstrap.cc:492] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a: Bootstrap starting.
I20260929 03:05:58.942270 16053 tablet_bootstrap.cc:492] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9: Bootstrap starting.
I20260929 03:05:58.943065 16051 tablet_bootstrap.cc:654] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:58.943873 16051 log.cc:824] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:58.944244 16053 tablet_bootstrap.cc:654] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:58.944571 16052 tablet_bootstrap.cc:492] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7: Bootstrap starting.
I20260929 03:05:58.945107 16053 log.cc:824] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:58.946300 16051 tablet_bootstrap.cc:492] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a: No bootstrap required, opened a new log
I20260929 03:05:58.946521 16051 ts_tablet_manager.cc:1397] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260929 03:05:58.946903 16053 tablet_bootstrap.cc:492] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9: No bootstrap required, opened a new log
I20260929 03:05:58.946954 16052 tablet_bootstrap.cc:654] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:58.947103 16053 ts_tablet_manager.cc:1397] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 03:05:58.947803 16052 log.cc:824] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7: Log is configured to *not* fsync() on all Append() calls
I20260929 03:05:58.949874 16052 tablet_bootstrap.cc:492] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7: No bootstrap required, opened a new log
I20260929 03:05:58.950065 16052 ts_tablet_manager.cc:1397] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260929 03:05:58.950634 16053 raft_consensus.cc:348] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } }
I20260929 03:05:58.950882 16053 raft_consensus.cc:374] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:58.950943 16053 raft_consensus.cc:729] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f74244291443d8a0c9fd12598681e9, State: Initialized, Role: FOLLOWER
I20260929 03:05:58.951401 16053 consensus_queue.cc:260] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } }
I20260929 03:05:58.951386 16051 raft_consensus.cc:348] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } }
I20260929 03:05:58.951659 16051 raft_consensus.cc:374] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:58.951716 16051 raft_consensus.cc:729] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 698950cba28047819f5cb12812d1ae2a, State: Initialized, Role: FOLLOWER
I20260929 03:05:58.952157 16023 heartbeater.cc:503] Master 127.7.209.126:45229 was elected leader, sending a full tablet report...
I20260929 03:05:58.952189 16051 consensus_queue.cc:260] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } }
I20260929 03:05:58.952528 16053 ts_tablet_manager.cc:1428] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 03:05:58.953168 16051 ts_tablet_manager.cc:1428] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260929 03:05:58.953593 15743 heartbeater.cc:503] Master 127.7.209.126:45229 was elected leader, sending a full tablet report...
I20260929 03:05:58.954025 16052 raft_consensus.cc:348] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } }
I20260929 03:05:58.954222 16052 raft_consensus.cc:374] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:58.954257 16052 raft_consensus.cc:729] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa1c916bdf44b60bb50948f340c83f7, State: Initialized, Role: FOLLOWER
I20260929 03:05:58.954644 16052 consensus_queue.cc:260] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } }
I20260929 03:05:58.955302 15883 heartbeater.cc:503] Master 127.7.209.126:45229 was elected leader, sending a full tablet report...
I20260929 03:05:58.955549 16052 ts_tablet_manager.cc:1428] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260929 03:05:58.985481 16058 raft_consensus.cc:482] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:58.985777 16058 raft_consensus.cc:504] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } }
I20260929 03:05:58.987255 16058 leader_election.cc:288] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213), 57f74244291443d8a0c9fd12598681e9 (127.7.209.67:35125)
I20260929 03:05:58.992607 16046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.987717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:54897 (local address 127.7.209.66:33213)
0929 03:05:58.987855 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.987860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.988060 (+ 200us) server_negotiation.cc:407] Connection header received
0929 03:05:58.988222 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.988226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.988286 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.988369 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:58.988714 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.989545 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.991342 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.991990 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.992097 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.992159 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.992377 (+ 218us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.992457 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 03:05:58.992607 16050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.988227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:49851 (local address 127.7.209.67:35125)
0929 03:05:58.988369 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:58.988374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:58.988512 (+ 138us) server_negotiation.cc:407] Connection header received
0929 03:05:58.988606 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:58.988608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:58.988657 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:58.988734 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:58.988958 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.989743 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.991339 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.991971 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.992097 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.992173 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:58.992377 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 03:05:58.992457 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 03:05:58.992654 16061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.988071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:35125 (local address 127.7.209.65:49851)
0929 03:05:58.988421 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:58.988432 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:58.988525 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:58.988757 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:58.988759 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:58.988775 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:58.988864 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.988868 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.989834 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.989837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.991216 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.991228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.992102 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.992106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.992235 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.992280 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 03:05:58.992355 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 03:05:58.992458 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":83,"thread_start_us":110,"threads_started":1}
I20260929 03:05:58.992654 16060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:58.987570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:33213 (local address 127.7.209.65:54897)
0929 03:05:58.987957 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:58.987973 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:58.988101 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:58.988408 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:58.988412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:58.988433 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:58.988585 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.988592 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.989714 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.989717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.991216 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:58.991228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:58.992102 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:58.992106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:58.992262 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:58.992282 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 03:05:58.992355 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 03:05:58.992458 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":133,"thread_start_us":155,"threads_started":1}
I20260929 03:05:58.993476 15977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e81163afdb44f1eb80e836b56e455a0" candidate_uuid: "698950cba28047819f5cb12812d1ae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f74244291443d8a0c9fd12598681e9" is_pre_election: true
I20260929 03:05:58.993475 15837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e81163afdb44f1eb80e836b56e455a0" candidate_uuid: "698950cba28047819f5cb12812d1ae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa1c916bdf44b60bb50948f340c83f7" is_pre_election: true
I20260929 03:05:58.993853 15837 raft_consensus.cc:2454] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 698950cba28047819f5cb12812d1ae2a in term 0.
I20260929 03:05:58.993852 15977 raft_consensus.cc:2454] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 698950cba28047819f5cb12812d1ae2a in term 0.
I20260929 03:05:58.994355 15628 leader_election.cc:302] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 698950cba28047819f5cb12812d1ae2a, caa1c916bdf44b60bb50948f340c83f7; no voters:
I20260929 03:05:58.994661 16058 raft_consensus.cc:2781] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:58.994768 16058 raft_consensus.cc:482] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:58.994815 16058 raft_consensus.cc:3037] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:58.996065 16058 raft_consensus.cc:504] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } }
I20260929 03:05:58.996390 16058 leader_election.cc:288] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [CANDIDATE]: Term 1 election: Requested vote from peers caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213), 57f74244291443d8a0c9fd12598681e9 (127.7.209.67:35125)
I20260929 03:05:58.996722 15837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e81163afdb44f1eb80e836b56e455a0" candidate_uuid: "698950cba28047819f5cb12812d1ae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa1c916bdf44b60bb50948f340c83f7"
I20260929 03:05:58.996865 15837 raft_consensus.cc:3037] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:58.996869 15977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e81163afdb44f1eb80e836b56e455a0" candidate_uuid: "698950cba28047819f5cb12812d1ae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f74244291443d8a0c9fd12598681e9"
I20260929 03:05:58.996989 15977 raft_consensus.cc:3037] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:58.998327 15977 raft_consensus.cc:2454] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 698950cba28047819f5cb12812d1ae2a in term 1.
I20260929 03:05:58.998436 15837 raft_consensus.cc:2454] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 698950cba28047819f5cb12812d1ae2a in term 1.
I20260929 03:05:58.998618 15628 leader_election.cc:302] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57f74244291443d8a0c9fd12598681e9, 698950cba28047819f5cb12812d1ae2a; no voters:
I20260929 03:05:58.998780 16058 raft_consensus.cc:2781] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:58.999022 16058 raft_consensus.cc:686] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 1 LEADER]: Becoming Leader. State: Replica: 698950cba28047819f5cb12812d1ae2a, State: Running, Role: LEADER
I20260929 03:05:58.999434 16058 consensus_queue.cc:237] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } }
I20260929 03:05:59.003216 15538 catalog_manager.cc:5495] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 698950cba28047819f5cb12812d1ae2a (127.7.209.65). New cstate: current_term: 1 leader_uuid: "698950cba28047819f5cb12812d1ae2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } health_report { overall_health: HEALTHY } } }
I20260929 03:05:59.026687 16049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.022747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54422 (local address 127.7.209.65:37335)
0929 03:05:59.022890 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:59.022894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:59.022911 (+ 17us) server_negotiation.cc:407] Connection header received
0929 03:05:59.023025 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:59.023028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:59.023122 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:59.023239 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:59.023513 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.024111 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.025042 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.025276 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.025372 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.025674 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:59.026273 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:59.026396 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:59.026455 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 03:05:59.026525 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 03:05:59.033946 15837 raft_consensus.cc:1261] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Refusing update from remote peer 698950cba28047819f5cb12812d1ae2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:59.034241 15977 raft_consensus.cc:1261] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Refusing update from remote peer 698950cba28047819f5cb12812d1ae2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:59.034791 16058 consensus_queue.cc:1035] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:59.035012 16062 consensus_queue.cc:1035] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:59.047056 16067 mvcc.cc:205] Tried to move back new op lower bound from 7334507147394363392 to 7334507147264983040. Current Snapshot: MvccSnapshot[applied={T|T < 7334507147394363392}]
I20260929 03:05:59.054556 15537 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:59.054765 15537 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:59.057301 15537 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b: Sending DeleteTablet for 3 replicas of tablet 4e81163afdb44f1eb80e836b56e455a0
I20260929 03:05:59.058131 15817 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e81163afdb44f1eb80e836b56e455a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:59 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:41904
I20260929 03:05:59.058131 15957 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e81163afdb44f1eb80e836b56e455a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:59 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:39724
I20260929 03:05:59.058616 15677 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e81163afdb44f1eb80e836b56e455a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:59 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:54418
I20260929 03:05:59.059010 16076 tablet_replica.cc:330] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9: stopping tablet replica
I20260929 03:05:59.059265 16075 tablet_replica.cc:330] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7: stopping tablet replica
I20260929 03:05:59.059523 16075 raft_consensus.cc:2229] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:59.059514 16076 raft_consensus.cc:2229] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:59.059494 15537 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 03:05:59.059880 16075 raft_consensus.cc:2258] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 03:05:59.059958 15537 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:59.060220 16076 raft_consensus.cc:2258] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:59.060601 16077 tablet_replica.cc:330] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a: stopping tablet replica
I20260929 03:05:59.060962 16077 raft_consensus.cc:2229] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:05:59.061396 16077 raft_consensus.cc:2258] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:59.062938 16076 ts_tablet_manager.cc:1905] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:59.062938 16077 ts_tablet_manager.cc:1905] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:59.063432 16075 ts_tablet_manager.cc:1905] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:59.065327 15677 tablet_service.cc:1467] Processing CreateTablet for tablet 9da6f0e82b98460e94b449a29058a458 (DEFAULT_TABLE table=test-table [id=36fcf45af9264264b4ebe9370dcfb088]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:59.065913 15817 tablet_service.cc:1467] Processing CreateTablet for tablet 9da6f0e82b98460e94b449a29058a458 (DEFAULT_TABLE table=test-table [id=36fcf45af9264264b4ebe9370dcfb088]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:59.066179 15817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da6f0e82b98460e94b449a29058a458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:59.066531 16076 ts_tablet_manager.cc:1918] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:59.066722 16076 log.cc:1196] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/4e81163afdb44f1eb80e836b56e455a0
I20260929 03:05:59.067216 16076 ts_tablet_manager.cc:1939] T 4e81163afdb44f1eb80e836b56e455a0 P 57f74244291443d8a0c9fd12598681e9: Deleting consensus metadata
I20260929 03:05:59.068135 16075 ts_tablet_manager.cc:1918] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:59.068239 16075 log.cc:1196] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/4e81163afdb44f1eb80e836b56e455a0
I20260929 03:05:59.068527 16052 tablet_bootstrap.cc:492] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7: Bootstrap starting.
I20260929 03:05:59.068539 16075 ts_tablet_manager.cc:1939] T 4e81163afdb44f1eb80e836b56e455a0 P caa1c916bdf44b60bb50948f340c83f7: Deleting consensus metadata
I20260929 03:05:59.068790 15957 tablet_service.cc:1467] Processing CreateTablet for tablet 9da6f0e82b98460e94b449a29058a458 (DEFAULT_TABLE table=test-table [id=36fcf45af9264264b4ebe9370dcfb088]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:59.069039 15957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da6f0e82b98460e94b449a29058a458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:59.069157 15677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9da6f0e82b98460e94b449a29058a458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:59.069574 16052 tablet_bootstrap.cc:654] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:59.070758 16077 ts_tablet_manager.cc:1918] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:05:59.070842 16077 log.cc:1196] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/wal/wals/4e81163afdb44f1eb80e836b56e455a0
I20260929 03:05:59.071113 16077 ts_tablet_manager.cc:1939] T 4e81163afdb44f1eb80e836b56e455a0 P 698950cba28047819f5cb12812d1ae2a: Deleting consensus metadata
I20260929 03:05:59.071421 16053 tablet_bootstrap.cc:492] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9: Bootstrap starting.
I20260929 03:05:59.071684 16051 tablet_bootstrap.cc:492] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a: Bootstrap starting.
I20260929 03:05:59.072293 15524 catalog_manager.cc:4841] TS caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213): tablet 4e81163afdb44f1eb80e836b56e455a0 (table test-table [id=4644ef8da8a142f58472e977a3f4e4fa]) successfully deleted
I20260929 03:05:59.072402 16053 tablet_bootstrap.cc:654] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:59.072469 15524 catalog_manager.cc:4841] TS 57f74244291443d8a0c9fd12598681e9 (127.7.209.67:35125): tablet 4e81163afdb44f1eb80e836b56e455a0 (table test-table [id=4644ef8da8a142f58472e977a3f4e4fa]) successfully deleted
I20260929 03:05:59.072641 16051 tablet_bootstrap.cc:654] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:59.073194 15525 catalog_manager.cc:4841] TS 698950cba28047819f5cb12812d1ae2a (127.7.209.65:37335): tablet 4e81163afdb44f1eb80e836b56e455a0 (table test-table [id=4644ef8da8a142f58472e977a3f4e4fa]) successfully deleted
I20260929 03:05:59.073771 16052 tablet_bootstrap.cc:492] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7: No bootstrap required, opened a new log
I20260929 03:05:59.073855 16052 ts_tablet_manager.cc:1397] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260929 03:05:59.074054 16053 tablet_bootstrap.cc:492] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9: No bootstrap required, opened a new log
I20260929 03:05:59.074136 16053 ts_tablet_manager.cc:1397] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 03:05:59.074311 16052 raft_consensus.cc:348] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.074415 16052 raft_consensus.cc:374] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:59.074458 16052 raft_consensus.cc:729] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa1c916bdf44b60bb50948f340c83f7, State: Initialized, Role: FOLLOWER
I20260929 03:05:59.074592 16052 consensus_queue.cc:260] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.074584 16053 raft_consensus.cc:348] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.074721 16053 raft_consensus.cc:374] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:59.074752 16053 raft_consensus.cc:729] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f74244291443d8a0c9fd12598681e9, State: Initialized, Role: FOLLOWER
I20260929 03:05:59.074862 16052 ts_tablet_manager.cc:1428] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 03:05:59.074862 16053 consensus_queue.cc:260] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.075094 16053 ts_tablet_manager.cc:1428] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:59.076612 16051 tablet_bootstrap.cc:492] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a: No bootstrap required, opened a new log
I20260929 03:05:59.076705 16051 ts_tablet_manager.cc:1397] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260929 03:05:59.077167 16051 raft_consensus.cc:348] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.077272 16051 raft_consensus.cc:374] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:59.077293 16051 raft_consensus.cc:729] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 698950cba28047819f5cb12812d1ae2a, State: Initialized, Role: FOLLOWER
I20260929 03:05:59.077394 16051 consensus_queue.cc:260] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.078012 16051 ts_tablet_manager.cc:1428] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260929 03:05:59.113236 16024 tablet.cc:2369] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:05:59.114606 15884 tablet.cc:2369] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:05:59.116914 15744 tablet.cc:2369] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:05:59.200991 16071 raft_consensus.cc:482] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:59.201138 16071 raft_consensus.cc:504] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.201633 16071 leader_election.cc:288] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213), 57f74244291443d8a0c9fd12598681e9 (127.7.209.67:35125)
I20260929 03:05:59.202169 15977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da6f0e82b98460e94b449a29058a458" candidate_uuid: "698950cba28047819f5cb12812d1ae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f74244291443d8a0c9fd12598681e9" is_pre_election: true
I20260929 03:05:59.202172 15837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da6f0e82b98460e94b449a29058a458" candidate_uuid: "698950cba28047819f5cb12812d1ae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa1c916bdf44b60bb50948f340c83f7" is_pre_election: true
I20260929 03:05:59.202343 15977 raft_consensus.cc:2454] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 698950cba28047819f5cb12812d1ae2a in term 0.
I20260929 03:05:59.202364 15837 raft_consensus.cc:2454] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 698950cba28047819f5cb12812d1ae2a in term 0.
I20260929 03:05:59.202697 15628 leader_election.cc:302] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 698950cba28047819f5cb12812d1ae2a, caa1c916bdf44b60bb50948f340c83f7; no voters:
I20260929 03:05:59.202886 16071 raft_consensus.cc:2781] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:59.202946 16071 raft_consensus.cc:482] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:59.202986 16071 raft_consensus.cc:3037] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:59.204223 16071 raft_consensus.cc:504] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.204577 16071 leader_election.cc:288] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [CANDIDATE]: Term 1 election: Requested vote from peers caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213), 57f74244291443d8a0c9fd12598681e9 (127.7.209.67:35125)
I20260929 03:05:59.205000 15837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da6f0e82b98460e94b449a29058a458" candidate_uuid: "698950cba28047819f5cb12812d1ae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa1c916bdf44b60bb50948f340c83f7"
I20260929 03:05:59.205052 15977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9da6f0e82b98460e94b449a29058a458" candidate_uuid: "698950cba28047819f5cb12812d1ae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f74244291443d8a0c9fd12598681e9"
I20260929 03:05:59.205157 15837 raft_consensus.cc:3037] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:59.205192 15977 raft_consensus.cc:3037] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:59.206327 15977 raft_consensus.cc:2454] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 698950cba28047819f5cb12812d1ae2a in term 1.
I20260929 03:05:59.206545 15837 raft_consensus.cc:2454] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 698950cba28047819f5cb12812d1ae2a in term 1.
I20260929 03:05:59.206669 15628 leader_election.cc:302] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57f74244291443d8a0c9fd12598681e9, 698950cba28047819f5cb12812d1ae2a; no voters:
I20260929 03:05:59.206950 16071 raft_consensus.cc:2781] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:59.207039 16071 raft_consensus.cc:686] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 1 LEADER]: Becoming Leader. State: Replica: 698950cba28047819f5cb12812d1ae2a, State: Running, Role: LEADER
I20260929 03:05:59.207206 16071 consensus_queue.cc:237] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.209419 15540 catalog_manager.cc:5495] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 698950cba28047819f5cb12812d1ae2a (127.7.209.65). New cstate: current_term: 1 leader_uuid: "698950cba28047819f5cb12812d1ae2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } health_report { overall_health: UNKNOWN } } }
Sep 29 03:05:59 dist-test-slave-bdps krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651157, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.126@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:59.269052 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.259996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:45229 (local address 127.0.0.1:56810)
0929 03:05:59.260338 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:59.260351 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:59.260491 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:59.260748 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:59.260751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:59.261297 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:05:59.261422 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:59.261429 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.262270 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.262274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:59.262865 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:59.262873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.263287 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.263290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:59.263422 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.264054 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:05:59.264068 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:05:59.265860 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:05:59.267408 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:59.267419 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:59.267429 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:59.267701 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:59.267948 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:05:59.267951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:05:59.267953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:05:59.268034 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:05:59.268401 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:05:59.268410 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:05:59.268535 (+ 125us) client_negotiation.cc:769] Sending connection context
0929 03:05:59.268635 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 03:05:59.268823 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":33,"thread_start_us":129,"threads_started":1}
I20260929 03:05:59.269052 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.260023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56810 (local address 127.7.209.126:45229)
0929 03:05:59.260154 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:59.260158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:59.260434 (+ 276us) server_negotiation.cc:407] Connection header received
0929 03:05:59.260576 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:59.260580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:59.260621 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:59.260709 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:59.261540 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.262163 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.262970 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.263187 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.263274 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.265986 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:59.265990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:59.265992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:59.266012 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:59.267313 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:59.267789 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:59.267793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:59.267795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:59.267872 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:59.268107 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:59.268110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:59.268112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:59.268303 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:59.268391 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:59.268700 (+ 309us) server_negotiation.cc:299] Negotiation successful
0929 03:05:59.268890 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":45}
I20260929 03:05:59.273777 15540 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:56810:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 03:05:59.274663 15540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:59.280025 15817 tablet_service.cc:1467] Processing CreateTablet for tablet f36eb72b432b4bc9b5d1411dbe91718a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=47cf72dfba72428bb226fcccbc8cdcfc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 03:05:59.279938 15677 tablet_service.cc:1467] Processing CreateTablet for tablet f36eb72b432b4bc9b5d1411dbe91718a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=47cf72dfba72428bb226fcccbc8cdcfc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 03:05:59.280054 15957 tablet_service.cc:1467] Processing CreateTablet for tablet f36eb72b432b4bc9b5d1411dbe91718a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=47cf72dfba72428bb226fcccbc8cdcfc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 03:05:59.280517 15957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f36eb72b432b4bc9b5d1411dbe91718a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:59.280519 15677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f36eb72b432b4bc9b5d1411dbe91718a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:59.280519 15817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f36eb72b432b4bc9b5d1411dbe91718a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:59.283667 16053 tablet_bootstrap.cc:492] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: Bootstrap starting.
I20260929 03:05:59.283735 16051 tablet_bootstrap.cc:492] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a: Bootstrap starting.
I20260929 03:05:59.283790 16052 tablet_bootstrap.cc:492] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7: Bootstrap starting.
I20260929 03:05:59.284746 16052 tablet_bootstrap.cc:654] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:59.284746 16051 tablet_bootstrap.cc:654] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:59.284868 16053 tablet_bootstrap.cc:654] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:59.286296 16052 tablet_bootstrap.cc:492] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7: No bootstrap required, opened a new log
I20260929 03:05:59.286301 16053 tablet_bootstrap.cc:492] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: No bootstrap required, opened a new log
I20260929 03:05:59.286301 16051 tablet_bootstrap.cc:492] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a: No bootstrap required, opened a new log
I20260929 03:05:59.286386 16051 ts_tablet_manager.cc:1397] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 03:05:59.286386 16053 ts_tablet_manager.cc:1397] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 03:05:59.286391 16052 ts_tablet_manager.cc:1397] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 03:05:59.286850 16052 raft_consensus.cc:348] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.286868 16051 raft_consensus.cc:348] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.286868 16053 raft_consensus.cc:348] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.286980 16051 raft_consensus.cc:374] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:59.286980 16052 raft_consensus.cc:374] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:59.286993 16053 raft_consensus.cc:374] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:59.287014 16052 raft_consensus.cc:729] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa1c916bdf44b60bb50948f340c83f7, State: Initialized, Role: FOLLOWER
I20260929 03:05:59.287014 16051 raft_consensus.cc:729] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 698950cba28047819f5cb12812d1ae2a, State: Initialized, Role: FOLLOWER
I20260929 03:05:59.287024 16053 raft_consensus.cc:729] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f74244291443d8a0c9fd12598681e9, State: Initialized, Role: FOLLOWER
I20260929 03:05:59.287158 16051 consensus_queue.cc:260] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.287166 16052 consensus_queue.cc:260] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.287158 16053 consensus_queue.cc:260] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.287446 16052 ts_tablet_manager.cc:1428] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:59.287451 16051 ts_tablet_manager.cc:1428] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:59.287451 16053 ts_tablet_manager.cc:1428] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:05:59.287431 16071 tablet_replica.cc:439] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } }
I20260929 03:05:59.287609 16071 tablet_replica.cc:442] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:59.287705 16059 tablet_replica.cc:439] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } }
I20260929 03:05:59.287709 16057 tablet_replica.cc:439] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } }
I20260929 03:05:59.287853 16059 tablet_replica.cc:442] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:59.287873 16057 tablet_replica.cc:442] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:59.551626 16057 raft_consensus.cc:482] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:05:59.551844 16057 raft_consensus.cc:504] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.553274 16057 leader_election.cc:288] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 698950cba28047819f5cb12812d1ae2a (127.7.209.65:37335), caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213)
I20260929 03:05:59.557446 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.553723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:53981 (local address 127.7.209.65:37335)
0929 03:05:59.554052 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:59.554057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:59.554075 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:05:59.554222 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:59.554225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:59.554275 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:59.554357 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:59.554654 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.555233 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.556359 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.556804 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.556911 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.556975 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:59.557203 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 03:05:59.557283 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":72,"threads_started":1}
I20260929 03:05:59.557446 16094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.553749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:53939 (local address 127.7.209.66:33213)
0929 03:05:59.554017 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:59.554023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:59.554102 (+ 79us) server_negotiation.cc:407] Connection header received
0929 03:05:59.554254 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:59.554257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:59.554305 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:59.554385 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:05:59.554682 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.555233 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.556371 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.556805 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.556913 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.556975 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:59.557201 (+ 226us) server_negotiation.cc:299] Negotiation successful
0929 03:05:59.557280 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":95,"threads_started":1}
I20260929 03:05:59.557448 16091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.553601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:33213 (local address 127.7.209.67:53939)
0929 03:05:59.554012 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:59.554026 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:59.554138 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:59.554400 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:59.554402 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:59.554446 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:59.554568 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:59.554573 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.555335 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.555339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:59.556257 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:59.556266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.556910 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.556913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:59.557034 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.557106 (+ 72us) client_negotiation.cc:769] Sending connection context
0929 03:05:59.557182 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 03:05:59.557264 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":81,"spinlock_wait_cycles":2688,"thread_start_us":82,"threads_started":1}
I20260929 03:05:59.557456 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.553572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:37335 (local address 127.7.209.67:53981)
0929 03:05:59.553948 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:59.553965 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:59.554101 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:59.554373 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:59.554375 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:59.554391 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:05:59.554554 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:59.554559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.555338 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.555341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:59.556246 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:59.556255 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.556910 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.556913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:59.557068 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.557108 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 03:05:59.557178 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 03:05:59.557265 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":105,"spinlock_wait_cycles":1920,"thread_start_us":82,"threads_started":1}
I20260929 03:05:59.558087 15697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36eb72b432b4bc9b5d1411dbe91718a" candidate_uuid: "57f74244291443d8a0c9fd12598681e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "698950cba28047819f5cb12812d1ae2a" is_pre_election: true
I20260929 03:05:59.558138 15837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36eb72b432b4bc9b5d1411dbe91718a" candidate_uuid: "57f74244291443d8a0c9fd12598681e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa1c916bdf44b60bb50948f340c83f7" is_pre_election: true
I20260929 03:05:59.558298 15837 raft_consensus.cc:2454] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57f74244291443d8a0c9fd12598681e9 in term 0.
I20260929 03:05:59.558367 15697 raft_consensus.cc:2454] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57f74244291443d8a0c9fd12598681e9 in term 0.
I20260929 03:05:59.558753 15908 leader_election.cc:302] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 57f74244291443d8a0c9fd12598681e9, 698950cba28047819f5cb12812d1ae2a, caa1c916bdf44b60bb50948f340c83f7; no voters:
I20260929 03:05:59.559034 16057 raft_consensus.cc:2781] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:05:59.559090 16057 raft_consensus.cc:482] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:05:59.559118 16057 raft_consensus.cc:3037] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:59.560381 16057 raft_consensus.cc:504] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.560748 16057 leader_election.cc:288] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [CANDIDATE]: Term 1 election: Requested vote from peers 698950cba28047819f5cb12812d1ae2a (127.7.209.65:37335), caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213)
I20260929 03:05:59.561085 15697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36eb72b432b4bc9b5d1411dbe91718a" candidate_uuid: "57f74244291443d8a0c9fd12598681e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "698950cba28047819f5cb12812d1ae2a"
I20260929 03:05:59.561085 15837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f36eb72b432b4bc9b5d1411dbe91718a" candidate_uuid: "57f74244291443d8a0c9fd12598681e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa1c916bdf44b60bb50948f340c83f7"
I20260929 03:05:59.561219 15837 raft_consensus.cc:3037] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:59.561251 15697 raft_consensus.cc:3037] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:05:59.562574 15697 raft_consensus.cc:2454] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57f74244291443d8a0c9fd12598681e9 in term 1.
I20260929 03:05:59.562574 15837 raft_consensus.cc:2454] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57f74244291443d8a0c9fd12598681e9 in term 1.
I20260929 03:05:59.562865 15909 leader_election.cc:302] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57f74244291443d8a0c9fd12598681e9, 698950cba28047819f5cb12812d1ae2a; no voters:
I20260929 03:05:59.563026 16057 raft_consensus.cc:2781] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:05:59.563220 16057 raft_consensus.cc:686] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [term 1 LEADER]: Becoming Leader. State: Replica: 57f74244291443d8a0c9fd12598681e9, State: Running, Role: LEADER
I20260929 03:05:59.563534 16057 consensus_queue.cc:237] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.564997 16095 tablet_replica.cc:439] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: TxnStatusTablet state changed. Reason: New leader 57f74244291443d8a0c9fd12598681e9. Latest consensus state: current_term: 1 leader_uuid: "57f74244291443d8a0c9fd12598681e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } }
I20260929 03:05:59.565135 16095 tablet_replica.cc:442] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: This TxnStatusTablet replica's current role is: LEADER
I20260929 03:05:59.565444 16097 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 03:05:59.565676 16097 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 03:05:59.566648 15537 catalog_manager.cc:5495] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57f74244291443d8a0c9fd12598681e9 (127.7.209.67). New cstate: current_term: 1 leader_uuid: "57f74244291443d8a0c9fd12598681e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } health_report { overall_health: HEALTHY } } }
I20260929 03:05:59.586084 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.581541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:35125 (local address 127.0.0.1:39726)
0929 03:05:59.581745 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:59.581765 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:59.581894 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:59.582431 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:59.582434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:59.582464 (+ 30us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 03:05:59.582595 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:59.582602 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.583422 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.583425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:59.584105 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:59.584113 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.584720 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.584724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:59.584827 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.584883 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 03:05:59.585740 (+ 857us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 03:05:59.585759 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 03:05:59.585827 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 03:05:59.585938 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 03:05:59.586105 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.581695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39726 (local address 127.7.209.67:35125)
0929 03:05:59.582132 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:59.582136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:59.582149 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:05:59.582231 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:59.582233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:59.582293 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:59.582397 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:59.582705 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.583315 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.584315 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.584587 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.584712 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.585034 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:59.585646 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:59.585776 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:59.585840 (+ 64us) server_negotiation.cc:299] Negotiation successful
0929 03:05:59.585906 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":112,"threads_started":1}
I20260929 03:05:59.590153 15837 raft_consensus.cc:1261] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Refusing update from remote peer 57f74244291443d8a0c9fd12598681e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:59.590350 15697 raft_consensus.cc:1261] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Refusing update from remote peer 57f74244291443d8a0c9fd12598681e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:59.590787 16095 consensus_queue.cc:1035] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:59.591054 16057 consensus_queue.cc:1035] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:59.592666 16059 tablet_replica.cc:439] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7: TxnStatusTablet state changed. Reason: New leader 57f74244291443d8a0c9fd12598681e9. Latest consensus state: current_term: 1 leader_uuid: "57f74244291443d8a0c9fd12598681e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } }
I20260929 03:05:59.592789 16059 tablet_replica.cc:442] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:59.593075 16071 tablet_replica.cc:439] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a: TxnStatusTablet state changed. Reason: New leader 57f74244291443d8a0c9fd12598681e9. Latest consensus state: current_term: 1 leader_uuid: "57f74244291443d8a0c9fd12598681e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } }
I20260929 03:05:59.593194 16071 tablet_replica.cc:442] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:59.595484 16095 tablet_replica.cc:439] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57f74244291443d8a0c9fd12598681e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } }
I20260929 03:05:59.595597 16095 tablet_replica.cc:442] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: This TxnStatusTablet replica's current role is: LEADER
I20260929 03:05:59.595736 16095 tablet_replica.cc:439] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57f74244291443d8a0c9fd12598681e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } }
I20260929 03:05:59.595827 16095 tablet_replica.cc:442] T f36eb72b432b4bc9b5d1411dbe91718a P 57f74244291443d8a0c9fd12598681e9: This TxnStatusTablet replica's current role is: LEADER
I20260929 03:05:59.596668 16059 tablet_replica.cc:439] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57f74244291443d8a0c9fd12598681e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } }
I20260929 03:05:59.596783 16059 tablet_replica.cc:442] T f36eb72b432b4bc9b5d1411dbe91718a P caa1c916bdf44b60bb50948f340c83f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:59.597457 16071 tablet_replica.cc:439] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57f74244291443d8a0c9fd12598681e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } } }
I20260929 03:05:59.597494 16103 mvcc.cc:205] Tried to move back new op lower bound from 7334507149674762240 to 7334507149575364608. Current Snapshot: MvccSnapshot[applied={T|T < 7334507149674762240}]
I20260929 03:05:59.597574 16071 tablet_replica.cc:442] T f36eb72b432b4bc9b5d1411dbe91718a P 698950cba28047819f5cb12812d1ae2a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 03:05:59.599367 16101 mvcc.cc:205] Tried to move back new op lower bound from 7334507149674762240 to 7334507149575364608. Current Snapshot: MvccSnapshot[applied={T|T < 7334507149674762240 or (T in {7334507149674762240})}]
I20260929 03:05:59.612116 16100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.608375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:59299 (local address 127.7.209.67:35125)
0929 03:05:59.608507 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:59.608511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:59.608808 (+ 297us) server_negotiation.cc:407] Connection header received
0929 03:05:59.608961 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:59.608964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:59.609021 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:59.609092 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:59.609349 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.609959 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.610992 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.611233 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.611327 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.611622 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:59.611718 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:59.611780 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:59.611881 (+ 101us) server_negotiation.cc:299] Negotiation successful
0929 03:05:59.611947 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 03:05:59.612116 16111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.608269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:35125 (local address 127.7.209.65:59299)
0929 03:05:59.608703 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:59.608718 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:59.608850 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:59.609108 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:59.609111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:59.609119 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 03:05:59.609248 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:59.609254 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.610071 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.610075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:59.610870 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:59.610881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.611343 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.611346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:59.611425 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.611482 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 03:05:59.611797 (+ 315us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 03:05:59.611803 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 03:05:59.611864 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 03:05:59.611974 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":131,"threads_started":1}
I20260929 03:05:59.621879 15977 raft_consensus.cc:1261] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Refusing update from remote peer 698950cba28047819f5cb12812d1ae2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:59.622051 15837 raft_consensus.cc:1261] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Refusing update from remote peer 698950cba28047819f5cb12812d1ae2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:05:59.622465 16058 consensus_queue.cc:1035] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:59.622772 16071 consensus_queue.cc:1035] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:05:59.650103 16093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.645305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:56255 (local address 127.7.209.65:37335)
0929 03:05:59.645430 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:59.645435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:59.645701 (+ 266us) server_negotiation.cc:407] Connection header received
0929 03:05:59.645861 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:59.645864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:59.645924 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:59.646000 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:05:59.646458 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.647325 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.648717 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.649007 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.649128 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.649532 (+ 404us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:05:59.649680 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:05:59.649759 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:59.649869 (+ 110us) server_negotiation.cc:299] Negotiation successful
0929 03:05:59.649941 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 03:05:59.650127 16116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.645235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:37335 (local address 127.7.209.67:56255)
0929 03:05:59.645574 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0929 03:05:59.645632 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 03:05:59.645751 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:05:59.646142 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:05:59.646146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:05:59.646155 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 03:05:59.646325 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:59.646332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.647467 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.647470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:59.648490 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:05:59.648502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.649158 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.649163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:05:59.649278 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.649344 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 03:05:59.649768 (+ 424us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 03:05:59.649775 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 03:05:59.649855 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 03:05:59.649953 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":127,"threads_started":1}
I20260929 03:05:59.672465 15540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:59.672650 15540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:05:59.674850 15540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b: Sending DeleteTablet for 3 replicas of tablet 9da6f0e82b98460e94b449a29058a458
I20260929 03:05:59.675410 15677 tablet_service.cc:1514] Processing DeleteTablet for tablet 9da6f0e82b98460e94b449a29058a458 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:59 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:54418
I20260929 03:05:59.675493 15817 tablet_service.cc:1514] Processing DeleteTablet for tablet 9da6f0e82b98460e94b449a29058a458 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:59 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:41904
I20260929 03:05:59.675879 16120 tablet_replica.cc:330] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7: stopping tablet replica
I20260929 03:05:59.676021 16120 raft_consensus.cc:2229] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:59.676134 16120 raft_consensus.cc:2258] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:59.676834 16120 ts_tablet_manager.cc:1905] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:59.680562 16120 ts_tablet_manager.cc:1918] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:05:59.680660 16120 log.cc:1196] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/9da6f0e82b98460e94b449a29058a458
I20260929 03:05:59.680943 16120 ts_tablet_manager.cc:1939] T 9da6f0e82b98460e94b449a29058a458 P caa1c916bdf44b60bb50948f340c83f7: Deleting consensus metadata
I20260929 03:05:59.680888 15957 tablet_service.cc:1514] Processing DeleteTablet for tablet 9da6f0e82b98460e94b449a29058a458 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:05:59 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:39724
I20260929 03:05:59.681293 16119 tablet_replica.cc:330] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a: stopping tablet replica
I20260929 03:05:59.681453 16119 raft_consensus.cc:2229] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:05:59.681723 16119 raft_consensus.cc:2258] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:59.681708 15524 catalog_manager.cc:4841] TS caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213): tablet 9da6f0e82b98460e94b449a29058a458 (table test-table [id=36fcf45af9264264b4ebe9370dcfb088]) successfully deleted
I20260929 03:05:59.681983 16122 tablet_replica.cc:330] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9: stopping tablet replica
I20260929 03:05:59.682113 16122 raft_consensus.cc:2229] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:05:59.682231 16122 raft_consensus.cc:2258] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:05:59.682966 16122 ts_tablet_manager.cc:1905] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:59.682966 16119 ts_tablet_manager.cc:1905] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:05:59.686091 16122 ts_tablet_manager.cc:1918] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:05:59.686264 16122 log.cc:1196] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/9da6f0e82b98460e94b449a29058a458
I20260929 03:05:59.686290 16119 ts_tablet_manager.cc:1918] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:05:59.686369 16119 log.cc:1196] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/wal/wals/9da6f0e82b98460e94b449a29058a458
I20260929 03:05:59.686699 16119 ts_tablet_manager.cc:1939] T 9da6f0e82b98460e94b449a29058a458 P 698950cba28047819f5cb12812d1ae2a: Deleting consensus metadata
I20260929 03:05:59.686697 16122 ts_tablet_manager.cc:1939] T 9da6f0e82b98460e94b449a29058a458 P 57f74244291443d8a0c9fd12598681e9: Deleting consensus metadata
I20260929 03:05:59.687532 15524 catalog_manager.cc:4841] TS 57f74244291443d8a0c9fd12598681e9 (127.7.209.67:35125): tablet 9da6f0e82b98460e94b449a29058a458 (table test-table [id=36fcf45af9264264b4ebe9370dcfb088]) successfully deleted
I20260929 03:05:59.687693 15525 catalog_manager.cc:4841] TS 698950cba28047819f5cb12812d1ae2a (127.7.209.65:37335): tablet 9da6f0e82b98460e94b449a29058a458 (table test-table [id=36fcf45af9264264b4ebe9370dcfb088]) successfully deleted
Sep 29 03:05:59 dist-test-slave-bdps krb5kdc[15495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 03:05:59 dist-test-slave-bdps krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:05:59 dist-test-slave-bdps krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 03:05:59.706539 15589 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:05:59.699419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56816 (local address 127.7.209.126:45229)
0929 03:05:59.699578 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:59.699584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:59.699744 (+ 160us) server_negotiation.cc:407] Connection header received
0929 03:05:59.699872 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:59.699875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:59.699940 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:59.700051 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:59.700794 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.701701 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.702586 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.702926 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.703045 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.706384 (+ 3339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56816: BlockingRecv error: recv got EOF from 127.0.0.1:56816 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 29 03:05:59 dist-test-slave-bdps krb5kdc[15495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM
I20260929 03:05:59.716926 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:05:59.708012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56828 (local address 127.7.209.126:45229)
0929 03:05:59.708124 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 03:05:59.708127 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:05:59.708216 (+ 89us) server_negotiation.cc:407] Connection header received
0929 03:05:59.708354 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:05:59.708356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:05:59.708395 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:05:59.708462 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:05:59.709188 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.709735 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.710868 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:05:59.711078 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:05:59.711151 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:05:59.713852 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:05:59.713856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:05:59.713858 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:05:59.713881 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:05:59.715336 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:59.715725 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:59.715728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:59.715730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:59.715804 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:05:59.716071 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:05:59.716075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:05:59.716077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:05:59.716290 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:05:59.716414 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 03:05:59.716628 (+ 214us) server_negotiation.cc:299] Negotiation successful
0929 03:05:59.716749 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 03:05:59.719600 15540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:05:59.720120 15540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:05:59.725427 15677 tablet_service.cc:1467] Processing CreateTablet for tablet 44f3282c443d4cbb89e0ce791714537f (DEFAULT_TABLE table=test-table [id=9aad56ff4422449abe58b22d9adaa62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:59.725477 15817 tablet_service.cc:1467] Processing CreateTablet for tablet 44f3282c443d4cbb89e0ce791714537f (DEFAULT_TABLE table=test-table [id=9aad56ff4422449abe58b22d9adaa62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:59.725875 15817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f3282c443d4cbb89e0ce791714537f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:59.725911 15957 tablet_service.cc:1467] Processing CreateTablet for tablet 44f3282c443d4cbb89e0ce791714537f (DEFAULT_TABLE table=test-table [id=9aad56ff4422449abe58b22d9adaa62b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:05:59.726230 15957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f3282c443d4cbb89e0ce791714537f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:59.728518 16052 tablet_bootstrap.cc:492] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7: Bootstrap starting.
I20260929 03:05:59.729461 15677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44f3282c443d4cbb89e0ce791714537f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:05:59.729562 16052 tablet_bootstrap.cc:654] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:59.731348 16052 tablet_bootstrap.cc:492] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7: No bootstrap required, opened a new log
I20260929 03:05:59.731447 16052 ts_tablet_manager.cc:1397] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 03:05:59.732172 16052 raft_consensus.cc:348] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.732285 16052 raft_consensus.cc:374] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:59.732314 16052 raft_consensus.cc:729] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa1c916bdf44b60bb50948f340c83f7, State: Initialized, Role: FOLLOWER
I20260929 03:05:59.732455 16052 consensus_queue.cc:260] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.732570 16053 tablet_bootstrap.cc:492] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9: Bootstrap starting.
I20260929 03:05:59.732740 16052 ts_tablet_manager.cc:1428] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 03:05:59.733572 16053 tablet_bootstrap.cc:654] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:59.734938 16053 tablet_bootstrap.cc:492] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9: No bootstrap required, opened a new log
I20260929 03:05:59.735033 16053 ts_tablet_manager.cc:1397] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 03:05:59.735455 16051 tablet_bootstrap.cc:492] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a: Bootstrap starting.
I20260929 03:05:59.735509 16053 raft_consensus.cc:348] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.735638 16053 raft_consensus.cc:374] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:59.735666 16053 raft_consensus.cc:729] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f74244291443d8a0c9fd12598681e9, State: Initialized, Role: FOLLOWER
I20260929 03:05:59.735795 16053 consensus_queue.cc:260] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.736070 16053 ts_tablet_manager.cc:1428] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 03:05:59.736406 16051 tablet_bootstrap.cc:654] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a: Neither blocks nor log segments found. Creating new log.
I20260929 03:05:59.737777 16051 tablet_bootstrap.cc:492] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a: No bootstrap required, opened a new log
I20260929 03:05:59.737875 16051 ts_tablet_manager.cc:1397] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260929 03:05:59.738324 16051 raft_consensus.cc:348] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.738473 16051 raft_consensus.cc:374] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:05:59.738516 16051 raft_consensus.cc:729] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 698950cba28047819f5cb12812d1ae2a, State: Initialized, Role: FOLLOWER
I20260929 03:05:59.738654 16051 consensus_queue.cc:260] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:05:59.739080 16051 ts_tablet_manager.cc:1428] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:06:00.093084 16095 raft_consensus.cc:482] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:06:00.093276 16095 raft_consensus.cc:504] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:06:00.093823 16095 leader_election.cc:288] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 698950cba28047819f5cb12812d1ae2a (127.7.209.65:37335), caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213)
I20260929 03:06:00.094360 15837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f3282c443d4cbb89e0ce791714537f" candidate_uuid: "57f74244291443d8a0c9fd12598681e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa1c916bdf44b60bb50948f340c83f7" is_pre_election: true
I20260929 03:06:00.094359 15697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f3282c443d4cbb89e0ce791714537f" candidate_uuid: "57f74244291443d8a0c9fd12598681e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "698950cba28047819f5cb12812d1ae2a" is_pre_election: true
I20260929 03:06:00.094555 15697 raft_consensus.cc:2454] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57f74244291443d8a0c9fd12598681e9 in term 0.
I20260929 03:06:00.094555 15837 raft_consensus.cc:2454] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57f74244291443d8a0c9fd12598681e9 in term 0.
I20260929 03:06:00.094889 15909 leader_election.cc:302] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57f74244291443d8a0c9fd12598681e9, 698950cba28047819f5cb12812d1ae2a; no voters:
I20260929 03:06:00.095074 16095 raft_consensus.cc:2781] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:06:00.095141 16095 raft_consensus.cc:482] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:06:00.095170 16095 raft_consensus.cc:3037] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:00.096395 16095 raft_consensus.cc:504] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:06:00.096705 16095 leader_election.cc:288] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [CANDIDATE]: Term 1 election: Requested vote from peers 698950cba28047819f5cb12812d1ae2a (127.7.209.65:37335), caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213)
I20260929 03:06:00.097056 15697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f3282c443d4cbb89e0ce791714537f" candidate_uuid: "57f74244291443d8a0c9fd12598681e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "698950cba28047819f5cb12812d1ae2a"
I20260929 03:06:00.097075 15837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44f3282c443d4cbb89e0ce791714537f" candidate_uuid: "57f74244291443d8a0c9fd12598681e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa1c916bdf44b60bb50948f340c83f7"
I20260929 03:06:00.097205 15837 raft_consensus.cc:3037] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:00.097210 15697 raft_consensus.cc:3037] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:00.098275 15837 raft_consensus.cc:2454] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57f74244291443d8a0c9fd12598681e9 in term 1.
I20260929 03:06:00.098351 15697 raft_consensus.cc:2454] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57f74244291443d8a0c9fd12598681e9 in term 1.
I20260929 03:06:00.098564 15908 leader_election.cc:302] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57f74244291443d8a0c9fd12598681e9, caa1c916bdf44b60bb50948f340c83f7; no voters:
I20260929 03:06:00.098713 16095 raft_consensus.cc:2781] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:06:00.098801 16095 raft_consensus.cc:686] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 1 LEADER]: Becoming Leader. State: Replica: 57f74244291443d8a0c9fd12598681e9, State: Running, Role: LEADER
I20260929 03:06:00.098950 16095 consensus_queue.cc:237] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:06:00.100732 15540 catalog_manager.cc:5495] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57f74244291443d8a0c9fd12598681e9 (127.7.209.67). New cstate: current_term: 1 leader_uuid: "57f74244291443d8a0c9fd12598681e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } health_report { overall_health: HEALTHY } } }
I20260929 03:06:00.183027 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:00.179708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39738 (local address 127.7.209.67:35125)
0929 03:06:00.179951 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:00.179954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:00.179968 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:06:00.180023 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:00.180025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:00.180076 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:00.180158 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:06:00.180399 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:00.180962 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:00.181850 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:00.182130 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:00.182232 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:00.182505 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:06:00.182597 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:06:00.182657 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:00.182777 (+ 120us) server_negotiation.cc:299] Negotiation successful
0929 03:06:00.182846 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":75,"threads_started":1}
I20260929 03:06:00.185837 15837 raft_consensus.cc:1261] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Refusing update from remote peer 57f74244291443d8a0c9fd12598681e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:06:00.185839 15697 raft_consensus.cc:1261] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Refusing update from remote peer 57f74244291443d8a0c9fd12598681e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:06:00.186304 16095 consensus_queue.cc:1035] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:06:00.186488 16112 consensus_queue.cc:1035] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:06:00.198072 15540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:06:00.198240 15540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:06:00.200304 15540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b: Sending DeleteTablet for 3 replicas of tablet 44f3282c443d4cbb89e0ce791714537f
I20260929 03:06:00.200901 15677 tablet_service.cc:1514] Processing DeleteTablet for tablet 44f3282c443d4cbb89e0ce791714537f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:06:00 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:54418
I20260929 03:06:00.201004 15817 tablet_service.cc:1514] Processing DeleteTablet for tablet 44f3282c443d4cbb89e0ce791714537f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:06:00 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:41904
I20260929 03:06:00.201192 15957 tablet_service.cc:1514] Processing DeleteTablet for tablet 44f3282c443d4cbb89e0ce791714537f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:06:00 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:39724
I20260929 03:06:00.201318 16151 tablet_replica.cc:330] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a: stopping tablet replica
I20260929 03:06:00.201438 16152 tablet_replica.cc:330] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7: stopping tablet replica
I20260929 03:06:00.201540 16151 raft_consensus.cc:2229] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:06:00.201560 16152 raft_consensus.cc:2229] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:06:00.201730 16152 raft_consensus.cc:2258] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:06:00.201838 16151 raft_consensus.cc:2258] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:06:00.202260 15540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 03:06:00.202696 16152 ts_tablet_manager.cc:1905] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 03:06:00.202737 15540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:06:00.202920 16151 ts_tablet_manager.cc:1905] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:06:00.206079 16152 ts_tablet_manager.cc:1918] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:06:00.206084 16151 ts_tablet_manager.cc:1918] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:06:00.206185 16152 log.cc:1196] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/44f3282c443d4cbb89e0ce791714537f
I20260929 03:06:00.206336 16151 log.cc:1196] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/wal/wals/44f3282c443d4cbb89e0ce791714537f
I20260929 03:06:00.206470 16152 ts_tablet_manager.cc:1939] T 44f3282c443d4cbb89e0ce791714537f P caa1c916bdf44b60bb50948f340c83f7: Deleting consensus metadata
I20260929 03:06:00.206813 16151 ts_tablet_manager.cc:1939] T 44f3282c443d4cbb89e0ce791714537f P 698950cba28047819f5cb12812d1ae2a: Deleting consensus metadata
I20260929 03:06:00.207842 16153 tablet_replica.cc:330] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9: stopping tablet replica
I20260929 03:06:00.208007 15525 catalog_manager.cc:4841] TS 698950cba28047819f5cb12812d1ae2a (127.7.209.65:37335): tablet 44f3282c443d4cbb89e0ce791714537f (table test-table [id=9aad56ff4422449abe58b22d9adaa62b]) successfully deleted
I20260929 03:06:00.208144 16153 raft_consensus.cc:2229] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:06:00.208556 15817 tablet_service.cc:1467] Processing CreateTablet for tablet e93d94a5938b4facb0dce8d7418fd02c (DEFAULT_TABLE table=test-table [id=0cebfd3d5410482a9b87839c52224b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:06:00.208633 15677 tablet_service.cc:1467] Processing CreateTablet for tablet e93d94a5938b4facb0dce8d7418fd02c (DEFAULT_TABLE table=test-table [id=0cebfd3d5410482a9b87839c52224b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:06:00.208832 16153 raft_consensus.cc:2258] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:06:00.208886 15817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93d94a5938b4facb0dce8d7418fd02c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:06:00.208904 15677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93d94a5938b4facb0dce8d7418fd02c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:06:00.209722 16153 ts_tablet_manager.cc:1905] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:06:00.209908 15957 tablet_service.cc:1467] Processing CreateTablet for tablet e93d94a5938b4facb0dce8d7418fd02c (DEFAULT_TABLE table=test-table [id=0cebfd3d5410482a9b87839c52224b27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:06:00.210165 15957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93d94a5938b4facb0dce8d7418fd02c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:06:00.211488 16051 tablet_bootstrap.cc:492] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a: Bootstrap starting.
I20260929 03:06:00.212391 16051 tablet_bootstrap.cc:654] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a: Neither blocks nor log segments found. Creating new log.
I20260929 03:06:00.212548 16053 tablet_bootstrap.cc:492] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9: Bootstrap starting.
I20260929 03:06:00.212929 16153 ts_tablet_manager.cc:1918] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:06:00.213057 16153 log.cc:1196] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/44f3282c443d4cbb89e0ce791714537f
I20260929 03:06:00.213403 16153 ts_tablet_manager.cc:1939] T 44f3282c443d4cbb89e0ce791714537f P 57f74244291443d8a0c9fd12598681e9: Deleting consensus metadata
I20260929 03:06:00.213470 16053 tablet_bootstrap.cc:654] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9: Neither blocks nor log segments found. Creating new log.
I20260929 03:06:00.213816 15524 catalog_manager.cc:4841] TS caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213): tablet 44f3282c443d4cbb89e0ce791714537f (table test-table [id=9aad56ff4422449abe58b22d9adaa62b]) successfully deleted
I20260929 03:06:00.214308 15524 catalog_manager.cc:4841] TS 57f74244291443d8a0c9fd12598681e9 (127.7.209.67:35125): tablet 44f3282c443d4cbb89e0ce791714537f (table test-table [id=9aad56ff4422449abe58b22d9adaa62b]) successfully deleted
I20260929 03:06:00.216115 16052 tablet_bootstrap.cc:492] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7: Bootstrap starting.
I20260929 03:06:00.217132 16052 tablet_bootstrap.cc:654] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7: Neither blocks nor log segments found. Creating new log.
I20260929 03:06:00.217218 16053 tablet_bootstrap.cc:492] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9: No bootstrap required, opened a new log
I20260929 03:06:00.217429 16051 tablet_bootstrap.cc:492] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a: No bootstrap required, opened a new log
I20260929 03:06:00.217511 16051 ts_tablet_manager.cc:1397] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260929 03:06:00.217902 16053 ts_tablet_manager.cc:1397] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260929 03:06:00.218463 16053 raft_consensus.cc:348] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:06:00.218595 16053 raft_consensus.cc:374] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:06:00.218623 16053 raft_consensus.cc:729] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f74244291443d8a0c9fd12598681e9, State: Initialized, Role: FOLLOWER
I20260929 03:06:00.218745 16053 consensus_queue.cc:260] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:06:00.219014 16053 ts_tablet_manager.cc:1428] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260929 03:06:00.219503 16051 raft_consensus.cc:348] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:06:00.219627 16051 raft_consensus.cc:374] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:06:00.219671 16051 raft_consensus.cc:729] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 698950cba28047819f5cb12812d1ae2a, State: Initialized, Role: FOLLOWER
I20260929 03:06:00.219795 16051 consensus_queue.cc:260] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:06:00.220556 16051 ts_tablet_manager.cc:1428] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260929 03:06:00.223389 16052 tablet_bootstrap.cc:492] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7: No bootstrap required, opened a new log
I20260929 03:06:00.223481 16052 ts_tablet_manager.cc:1397] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260929 03:06:00.223929 16052 raft_consensus.cc:348] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:06:00.224035 16052 raft_consensus.cc:374] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:06:00.224063 16052 raft_consensus.cc:729] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa1c916bdf44b60bb50948f340c83f7, State: Initialized, Role: FOLLOWER
I20260929 03:06:00.224190 16052 consensus_queue.cc:260] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:06:00.224442 16052 ts_tablet_manager.cc:1428] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260929 03:06:00.311733 16058 raft_consensus.cc:482] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:06:00.311969 16058 raft_consensus.cc:504] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:06:00.312601 16058 leader_election.cc:288] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213), 57f74244291443d8a0c9fd12598681e9 (127.7.209.67:35125)
I20260929 03:06:00.313205 15977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93d94a5938b4facb0dce8d7418fd02c" candidate_uuid: "698950cba28047819f5cb12812d1ae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f74244291443d8a0c9fd12598681e9" is_pre_election: true
I20260929 03:06:00.313205 15837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93d94a5938b4facb0dce8d7418fd02c" candidate_uuid: "698950cba28047819f5cb12812d1ae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa1c916bdf44b60bb50948f340c83f7" is_pre_election: true
I20260929 03:06:00.313397 15977 raft_consensus.cc:2454] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 698950cba28047819f5cb12812d1ae2a in term 0.
I20260929 03:06:00.313397 15837 raft_consensus.cc:2454] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 698950cba28047819f5cb12812d1ae2a in term 0.
I20260929 03:06:00.313798 15628 leader_election.cc:302] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57f74244291443d8a0c9fd12598681e9, 698950cba28047819f5cb12812d1ae2a; no voters:
I20260929 03:06:00.314073 16058 raft_consensus.cc:2781] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:06:00.314150 16058 raft_consensus.cc:482] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:06:00.314205 16058 raft_consensus.cc:3037] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:00.315991 16058 raft_consensus.cc:504] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:06:00.316417 16058 leader_election.cc:288] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [CANDIDATE]: Term 1 election: Requested vote from peers caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213), 57f74244291443d8a0c9fd12598681e9 (127.7.209.67:35125)
I20260929 03:06:00.316872 15837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93d94a5938b4facb0dce8d7418fd02c" candidate_uuid: "698950cba28047819f5cb12812d1ae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caa1c916bdf44b60bb50948f340c83f7"
I20260929 03:06:00.316895 15977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93d94a5938b4facb0dce8d7418fd02c" candidate_uuid: "698950cba28047819f5cb12812d1ae2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f74244291443d8a0c9fd12598681e9"
I20260929 03:06:00.317015 15977 raft_consensus.cc:3037] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:00.317016 15837 raft_consensus.cc:3037] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:00.318360 15837 raft_consensus.cc:2454] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 698950cba28047819f5cb12812d1ae2a in term 1.
I20260929 03:06:00.318359 15977 raft_consensus.cc:2454] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 698950cba28047819f5cb12812d1ae2a in term 1.
I20260929 03:06:00.318683 15628 leader_election.cc:302] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 698950cba28047819f5cb12812d1ae2a, caa1c916bdf44b60bb50948f340c83f7; no voters:
I20260929 03:06:00.318929 16058 raft_consensus.cc:2781] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:06:00.319087 16058 raft_consensus.cc:686] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 1 LEADER]: Becoming Leader. State: Replica: 698950cba28047819f5cb12812d1ae2a, State: Running, Role: LEADER
I20260929 03:06:00.319257 16058 consensus_queue.cc:237] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } }
I20260929 03:06:00.321200 15538 catalog_manager.cc:5495] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 698950cba28047819f5cb12812d1ae2a (127.7.209.65). New cstate: current_term: 1 leader_uuid: "698950cba28047819f5cb12812d1ae2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "698950cba28047819f5cb12812d1ae2a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 } health_report { overall_health: UNKNOWN } } }
I20260929 03:06:00.354276 16159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:00.351261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54430 (local address 127.7.209.65:37335)
0929 03:06:00.351591 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:00.351598 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:00.351611 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:06:00.351650 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:00.351652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:00.351698 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:00.351774 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:06:00.351965 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:00.352537 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:00.353327 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:00.353560 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:00.353673 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:00.353864 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:06:00.353949 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:06:00.353999 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:00.354066 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 03:06:00.354119 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":161,"threads_started":1}
I20260929 03:06:00.360587 15977 raft_consensus.cc:1261] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Refusing update from remote peer 698950cba28047819f5cb12812d1ae2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:06:00.360587 15837 raft_consensus.cc:1261] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Refusing update from remote peer 698950cba28047819f5cb12812d1ae2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:06:00.361055 16058 consensus_queue.cc:1035] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "57f74244291443d8a0c9fd12598681e9" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:06:00.361249 16157 consensus_queue.cc:1035] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "caa1c916bdf44b60bb50948f340c83f7" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:06:00.398239 15540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:06:00.398411 15540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:06:00.400264 15540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ec35d50a35144cc8c0dc32690c4dd1b: Sending DeleteTablet for 3 replicas of tablet e93d94a5938b4facb0dce8d7418fd02c
I20260929 03:06:00.400729 15677 tablet_service.cc:1514] Processing DeleteTablet for tablet e93d94a5938b4facb0dce8d7418fd02c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:06:00 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:54418
I20260929 03:06:00.400929 16151 tablet_replica.cc:330] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a: stopping tablet replica
I20260929 03:06:00.400890 15817 tablet_service.cc:1514] Processing DeleteTablet for tablet e93d94a5938b4facb0dce8d7418fd02c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:06:00 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:41904
I20260929 03:06:00.400981 15957 tablet_service.cc:1514] Processing DeleteTablet for tablet e93d94a5938b4facb0dce8d7418fd02c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:06:00 UTC) from {username='oryx', principal='oryx/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:39724
I20260929 03:06:00.401049 16151 raft_consensus.cc:2229] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:06:00.401166 16153 tablet_replica.cc:330] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9: stopping tablet replica
I20260929 03:06:00.401175 16152 tablet_replica.cc:330] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7: stopping tablet replica
I20260929 03:06:00.401238 16151 raft_consensus.cc:2258] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:06:00.401284 16153 raft_consensus.cc:2229] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:06:00.401283 16152 raft_consensus.cc:2229] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:06:00.401394 16152 raft_consensus.cc:2258] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:06:00.401394 16153 raft_consensus.cc:2258] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:06:00.402165 16153 ts_tablet_manager.cc:1905] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:06:00.402230 16151 ts_tablet_manager.cc:1905] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:06:00.405176 16151 ts_tablet_manager.cc:1918] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:06:00.405179 16153 ts_tablet_manager.cc:1918] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:06:00.405272 16153 log.cc:1196] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/e93d94a5938b4facb0dce8d7418fd02c
I20260929 03:06:00.405272 16151 log.cc:1196] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-0/wal/wals/e93d94a5938b4facb0dce8d7418fd02c
I20260929 03:06:00.405566 16151 ts_tablet_manager.cc:1939] T e93d94a5938b4facb0dce8d7418fd02c P 698950cba28047819f5cb12812d1ae2a: Deleting consensus metadata
I20260929 03:06:00.405843 16152 ts_tablet_manager.cc:1905] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:06:00.406144 16153 ts_tablet_manager.cc:1939] T e93d94a5938b4facb0dce8d7418fd02c P 57f74244291443d8a0c9fd12598681e9: Deleting consensus metadata
I20260929 03:06:00.406425 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 15612
I20260929 03:06:00.406438 15525 catalog_manager.cc:4841] TS 698950cba28047819f5cb12812d1ae2a (127.7.209.65:37335): tablet e93d94a5938b4facb0dce8d7418fd02c (table test-table [id=0cebfd3d5410482a9b87839c52224b27]) successfully deleted
I20260929 03:06:00.408466 15524 catalog_manager.cc:4841] TS 57f74244291443d8a0c9fd12598681e9 (127.7.209.67:35125): tablet e93d94a5938b4facb0dce8d7418fd02c (table test-table [id=0cebfd3d5410482a9b87839c52224b27]) successfully deleted
I20260929 03:06:00.408635 16152 ts_tablet_manager.cc:1918] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 03:06:00.408713 16152 log.cc:1196] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/e93d94a5938b4facb0dce8d7418fd02c
I20260929 03:06:00.408962 16152 ts_tablet_manager.cc:1939] T e93d94a5938b4facb0dce8d7418fd02c P caa1c916bdf44b60bb50948f340c83f7: Deleting consensus metadata
I20260929 03:06:00.409602 15524 catalog_manager.cc:4841] TS caa1c916bdf44b60bb50948f340c83f7 (127.7.209.66:33213): tablet e93d94a5938b4facb0dce8d7418fd02c (table test-table [id=0cebfd3d5410482a9b87839c52224b27]) successfully deleted
I20260929 03:06:00.415344 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 15752
I20260929 03:06:00.421046 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 15892
I20260929 03:06:00.427222 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 15508
2026-09-29T03:06:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3669 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:06:00 dist-test-slave-bdps krb5kdc[16167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:06:00 dist-test-slave-bdps krb5kdc[16167](info): set up 2 sockets
Sep 29 03:06:00 dist-test-slave-bdps krb5kdc[16167](info): commencing operation
krb5kdc: starting...
W20260929 03:06:01.394568 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.944s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:06:01 dist-test-slave-bdps krb5kdc[16167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:06:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:06:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:06:01.520741 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:43333
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:43333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:06:01.644403 16180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:01.644734 16180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:01.644793 16180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:01.649024 16180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:06:01.649117 16180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:01.649147 16180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:01.649168 16180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:06:01.649192 16180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:06:01.655141 16180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:43333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:43333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:01.656576 16180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:01.657915 16180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:01.667574 16186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:01.667603 16185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:01.667574 16188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:01.667729 16180 server_base.cc:1034] running on GCE node
I20260929 03:06:01.668641 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:01.670087 16180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:01.671262 16180 hybrid_clock.cc:648] HybridClock initialized: now 1790651161671243 us; error 36 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T03:06:01Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1262 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:06:01 dist-test-slave-bdps krb5kdc[16193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:06:01 dist-test-slave-bdps krb5kdc[16193](info): set up 2 sockets
Sep 29 03:06:01 dist-test-slave-bdps krb5kdc[16193](info): commencing operation
krb5kdc: starting...
W20260929 03:06:02.646548 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:06:02 dist-test-slave-bdps krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:06:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:06:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:06:02.770922 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:38051
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:38051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:06:02.895779 16206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:02.896162 16206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:02.896209 16206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:02.900867 16206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:06:02.900959 16206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:02.900977 16206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:02.900995 16206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:06:02.901010 16206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:06:02.906847 16206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:38051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:38051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:02.908277 16206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:02.909539 16206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:02.919692 16212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:02.919692 16211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:02.919787 16206 server_base.cc:1034] running on GCE node
W20260929 03:06:02.919700 16214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:02.920809 16206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:02.922377 16206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:02.923578 16206 hybrid_clock.cc:648] HybridClock initialized: now 1790651162923560 us; error 35 us; skew 500 ppm
Sep 29 03:06:02 dist-test-slave-bdps krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651162, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:06:02.926928 16206 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:06:02.928113 16206 webserver.cc:460] Webserver started at http://127.7.209.126:32771/ using document root <none> and password file <none>
I20260929 03:06:02.928901 16206 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:02.928984 16206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:02.929255 16206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:02.932054 16206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "21469e7427f54891b23e0e9cd6774fa3"
format_stamp: "Formatted at 2026-09-29 03:06:02 on dist-test-slave-bdps"
server_key: "4e1c15c7a722c72720ae11f4f8d0704a"
server_key_iv: "a646f02095732103b8b7bc3535aae960"
server_key_version: "encryptionkey@0"
I20260929 03:06:02.932729 16206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "21469e7427f54891b23e0e9cd6774fa3"
format_stamp: "Formatted at 2026-09-29 03:06:02 on dist-test-slave-bdps"
server_key: "4e1c15c7a722c72720ae11f4f8d0704a"
server_key_iv: "a646f02095732103b8b7bc3535aae960"
server_key_version: "encryptionkey@0"
I20260929 03:06:02.937269 16206 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 03:06:02.940116 16221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:02.941581 16206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:06:02.941725 16206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "21469e7427f54891b23e0e9cd6774fa3"
format_stamp: "Formatted at 2026-09-29 03:06:02 on dist-test-slave-bdps"
server_key: "4e1c15c7a722c72720ae11f4f8d0704a"
server_key_iv: "a646f02095732103b8b7bc3535aae960"
server_key_version: "encryptionkey@0"
I20260929 03:06:02.941826 16206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:02.974941 16206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:02.978535 16206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:02.978753 16206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:02.986780 16206 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:38051
I20260929 03:06:02.986799 16273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:38051 every 8 connection(s)
I20260929 03:06:02.987931 16206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:06:02.988528 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 16206
I20260929 03:06:02.988686 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:06:02.989044 8005 external_mini_cluster.cc:1428] Setting key 64363fed8d08ed0d0a843bded2fa5a60
I20260929 03:06:02.991760 16274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:06:02.998601 16274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: Bootstrap starting.
Sep 29 03:06:02 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:06:03.001394 16274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: Neither blocks nor log segments found. Creating new log.
I20260929 03:06:03.002307 16274 log.cc:824] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: Log is configured to *not* fsync() on all Append() calls
I20260929 03:06:03.004520 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:02.990789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46062 (local address 127.7.209.126:38051)
0929 03:06:02.991280 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:02.991297 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:02.991341 (+ 44us) server_negotiation.cc:407] Connection header received
0929 03:06:02.992102 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:02.992124 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:02.992521 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:02.992908 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:02.993882 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:02.995007 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:02.995970 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:02.996517 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:02.996677 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:02.999514 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:02.999520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:02.999535 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:06:02.999566 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:03.001308 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:03.001844 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:03.001855 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:03.001860 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:03.001988 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:03.002284 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:03.002286 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:03.002287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:03.002791 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:03.002987 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:03.003311 (+ 324us) server_negotiation.cc:299] Negotiation successful
0929 03:06:03.003642 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":163,"threads_started":1}
I20260929 03:06:03.004570 16274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: No bootstrap required, opened a new log
I20260929 03:06:03.009187 16274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21469e7427f54891b23e0e9cd6774fa3" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38051 } }
I20260929 03:06:03.009555 16274 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:06:03.009646 16274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21469e7427f54891b23e0e9cd6774fa3, State: Initialized, Role: FOLLOWER
I20260929 03:06:03.010138 16274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21469e7427f54891b23e0e9cd6774fa3" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38051 } }
I20260929 03:06:03.010272 16274 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:06:03.010319 16274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:06:03.010399 16274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:03.011511 16274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21469e7427f54891b23e0e9cd6774fa3" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38051 } }
I20260929 03:06:03.011902 16274 leader_election.cc:302] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21469e7427f54891b23e0e9cd6774fa3; no voters:
I20260929 03:06:03.012220 16274 leader_election.cc:288] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:06:03.012357 16279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:06:03.012609 16279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 1 LEADER]: Becoming Leader. State: Replica: 21469e7427f54891b23e0e9cd6774fa3, State: Running, Role: LEADER
I20260929 03:06:03.012953 16279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21469e7427f54891b23e0e9cd6774fa3" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38051 } }
I20260929 03:06:03.013639 16274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:06:03.014878 16281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21469e7427f54891b23e0e9cd6774fa3. Latest consensus state: current_term: 1 leader_uuid: "21469e7427f54891b23e0e9cd6774fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21469e7427f54891b23e0e9cd6774fa3" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38051 } } }
I20260929 03:06:03.014833 16280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21469e7427f54891b23e0e9cd6774fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21469e7427f54891b23e0e9cd6774fa3" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38051 } } }
I20260929 03:06:03.015018 16281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [sys.catalog]: This master's current role is: LEADER
I20260929 03:06:03.015018 16280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [sys.catalog]: This master's current role is: LEADER
I20260929 03:06:03.015509 16288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:06:03.018962 16288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:06:03.025177 16288 catalog_manager.cc:1347] Generated new cluster ID: 40cc8917791d49e4865845269497c507
I20260929 03:06:03.025239 16288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:06:03.045776 16288 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:06:03.047319 16288 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:06:03.058269 16288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: Generated new TSK 0
I20260929 03:06:03.059289 16288 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:06:03.092914 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:38051
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:06:03.221553 16300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:03.221882 16300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:03.221932 16300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:03.226015 16300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:03.226073 16300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:03.226143 16300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:06:03.232051 16300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:38051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:03.233628 16300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:03.234962 16300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:03.245697 16306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:03.245705 16305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:03.245705 16308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:03.245810 16300 server_base.cc:1034] running on GCE node
I20260929 03:06:03.246464 16300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:03.247177 16300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:03.248340 16300 hybrid_clock.cc:648] HybridClock initialized: now 1790651163248307 us; error 44 us; skew 500 ppm
Sep 29 03:06:03 dist-test-slave-bdps krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651163, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:06:03.251569 16300 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:06:03.252601 16300 webserver.cc:460] Webserver started at http://127.7.209.65:46731/ using document root <none> and password file <none>
I20260929 03:06:03.253222 16300 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:03.253283 16300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:03.253482 16300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:03.255941 16300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "6008309c90e74d8186634e11c37c1670"
format_stamp: "Formatted at 2026-09-29 03:06:03 on dist-test-slave-bdps"
server_key: "5cefe7a502366bd9ac56f95afa813231"
server_key_iv: "7d24b6e0da67316182342391cb96843a"
server_key_version: "encryptionkey@0"
I20260929 03:06:03.256546 16300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "6008309c90e74d8186634e11c37c1670"
format_stamp: "Formatted at 2026-09-29 03:06:03 on dist-test-slave-bdps"
server_key: "5cefe7a502366bd9ac56f95afa813231"
server_key_iv: "7d24b6e0da67316182342391cb96843a"
server_key_version: "encryptionkey@0"
I20260929 03:06:03.260913 16300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 03:06:03.263592 16315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:03.264937 16300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 03:06:03.265050 16300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "6008309c90e74d8186634e11c37c1670"
format_stamp: "Formatted at 2026-09-29 03:06:03 on dist-test-slave-bdps"
server_key: "5cefe7a502366bd9ac56f95afa813231"
server_key_iv: "7d24b6e0da67316182342391cb96843a"
server_key_version: "encryptionkey@0"
I20260929 03:06:03.265147 16300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:03.277834 16300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:03.281224 16300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:03.281410 16300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:03.282140 16300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:03.283093 16300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:03.283140 16300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:03.283200 16300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:03.283231 16300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:03.293286 16300 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:38557
I20260929 03:06:03.293310 16428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:38557 every 8 connection(s)
I20260929 03:06:03.294418 16300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:06:03.301451 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 16300
I20260929 03:06:03.301637 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:06:03.301988 8005 external_mini_cluster.cc:1428] Setting key 76c5cd8f281c41f3867cd370d0ab181b
Sep 29 03:06:03 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651163, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:06:03.309423 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:03.296537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:57479 (local address 127.7.209.126:38051)
0929 03:06:03.296693 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:03.296697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:03.297688 (+ 991us) server_negotiation.cc:407] Connection header received
0929 03:06:03.299163 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:03.299185 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:03.299256 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:03.299338 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:03.300692 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:03.301373 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:03.302747 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:03.303054 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:03.303157 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:03.306027 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:03.306030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:03.306032 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:06:03.306055 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:03.307451 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:03.307944 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:03.307947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:03.307949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:03.308025 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:03.308340 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:03.308343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:03.308344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:03.308519 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:03.308604 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:03.309104 (+ 500us) server_negotiation.cc:299] Negotiation successful
0929 03:06:03.309247 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 03:06:03.310312 16431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:03.296985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:57479)
0929 03:06:03.297494 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:03.297544 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:03.298908 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:03.299548 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:03.299556 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:03.299962 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:03.300530 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:03.300546 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:03.301511 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:03.301514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:03.302612 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:03.302622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:03.303192 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:03.303195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:03.303381 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:03.304076 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:03.304096 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:03.305871 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:06:03.307558 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:06:03.307564 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:06:03.307591 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:06:03.307844 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:06:03.308104 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:06:03.308107 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:06:03.308110 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:06:03.308246 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:06:03.308613 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:06:03.308620 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:06:03.308874 (+ 254us) client_negotiation.cc:769] Sending connection context
0929 03:06:03.309036 (+ 162us) client_negotiation.cc:240] Negotiation successful
0929 03:06:03.309363 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":125,"threads_started":1}
I20260929 03:06:03.311633 16429 heartbeater.cc:347] Connected to a master server at 127.7.209.126:38051
I20260929 03:06:03.311936 16429 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:06:03.312438 16429 heartbeater.cc:511] Master 127.7.209.126:38051 requested a full tablet report, sending...
I20260929 03:06:03.314148 16238 ts_manager.cc:194] Registered new tserver with Master: 6008309c90e74d8186634e11c37c1670 (127.7.209.65:38557)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
I20260929 03:06:03.315671 16238 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:57479
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:06:03.330041 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:38051
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:06:03.456748 16434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:03.457074 16434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:03.457140 16434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:03.461259 16434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:03.461336 16434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:03.461423 16434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:06:03.467084 16434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:38051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:03.468429 16434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:03.469789 16434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:03.480400 16440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:03.480468 16434 server_base.cc:1034] running on GCE node
W20260929 03:06:03.480400 16439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:03.480458 16442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:03.481037 16434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:03.481783 16434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:03.482947 16434 hybrid_clock.cc:648] HybridClock initialized: now 1790651163482927 us; error 32 us; skew 500 ppm
Sep 29 03:06:03 dist-test-slave-bdps krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651163, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:06:03.486039 16434 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:06:03.487118 16434 webserver.cc:460] Webserver started at http://127.7.209.66:43917/ using document root <none> and password file <none>
I20260929 03:06:03.487824 16434 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:03.487898 16434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:03.488143 16434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:03.490772 16434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "6eb87f3418a2427db4285b623a9925c2"
format_stamp: "Formatted at 2026-09-29 03:06:03 on dist-test-slave-bdps"
server_key: "57d6b8d4d95616e7358c0666ac982ef5"
server_key_iv: "64d8cf47eb7eff5960ebe78bc86d4e11"
server_key_version: "encryptionkey@0"
I20260929 03:06:03.491439 16434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "6eb87f3418a2427db4285b623a9925c2"
format_stamp: "Formatted at 2026-09-29 03:06:03 on dist-test-slave-bdps"
server_key: "57d6b8d4d95616e7358c0666ac982ef5"
server_key_iv: "64d8cf47eb7eff5960ebe78bc86d4e11"
server_key_version: "encryptionkey@0"
I20260929 03:06:03.495913 16434 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:06:03.498718 16449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:03.500202 16434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:06:03.500312 16434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "6eb87f3418a2427db4285b623a9925c2"
format_stamp: "Formatted at 2026-09-29 03:06:03 on dist-test-slave-bdps"
server_key: "57d6b8d4d95616e7358c0666ac982ef5"
server_key_iv: "64d8cf47eb7eff5960ebe78bc86d4e11"
server_key_version: "encryptionkey@0"
I20260929 03:06:03.500416 16434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:03.517308 16434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:03.520853 16434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:03.521044 16434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:03.521770 16434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:03.522738 16434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:03.522789 16434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:03.522845 16434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:03.522873 16434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:03.532765 16434 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:43559
I20260929 03:06:03.532785 16562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:43559 every 8 connection(s)
I20260929 03:06:03.533941 16434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:06:03.537104 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 16434
I20260929 03:06:03.537272 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:06:03.537708 8005 external_mini_cluster.cc:1428] Setting key 7dfc92fef37c3ccd1fa62c4c86b204df
Sep 29 03:06:03 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651163, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:06:03.550392 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:03.536358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:59051 (local address 127.7.209.126:38051)
0929 03:06:03.536510 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:03.536515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:03.537653 (+ 1138us) server_negotiation.cc:407] Connection header received
0929 03:06:03.539202 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:03.539205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:03.539264 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:03.539343 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:03.540740 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:03.541315 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:03.543053 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:03.543296 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:03.543387 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:03.546760 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:03.546764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:03.546766 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:06:03.546792 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:03.548193 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:03.548804 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:03.548807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:03.548809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:03.548891 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:03.549255 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:03.549258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:03.549260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:03.549460 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:03.549536 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:03.550089 (+ 553us) server_negotiation.cc:299] Negotiation successful
0929 03:06:03.550206 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 03:06:03.551506 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:03.536843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:59051)
0929 03:06:03.537409 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:03.537475 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:03.538960 (+ 1485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:03.539557 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:03.539565 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:03.539980 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:03.540583 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:03.540600 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:03.541444 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:03.541448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:03.542927 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:03.542938 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:03.543463 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:03.543467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:03.543681 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:03.544406 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:03.544428 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:03.546564 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:06:03.548308 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:06:03.548316 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:06:03.548336 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:06:03.548700 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:06:03.548967 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:06:03.548971 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:06:03.548973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:06:03.549158 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:06:03.549551 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:06:03.549557 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:06:03.549873 (+ 316us) client_negotiation.cc:769] Sending connection context
0929 03:06:03.550044 (+ 171us) client_negotiation.cc:240] Negotiation successful
0929 03:06:03.550408 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
I20260929 03:06:03.552881 16563 heartbeater.cc:347] Connected to a master server at 127.7.209.126:38051
I20260929 03:06:03.553181 16563 heartbeater.cc:464] Registering TS with master...
I20260929 03:06:03.553894 16563 heartbeater.cc:511] Master 127.7.209.126:38051 requested a full tablet report, sending...
I20260929 03:06:03.555116 16238 ts_manager.cc:194] Registered new tserver with Master: 6eb87f3418a2427db4285b623a9925c2 (127.7.209.66:43559)
I20260929 03:06:03.555749 16238 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:59051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:06:03.568045 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:38051
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:06:03.692261 16568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:03.692657 16568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:03.692735 16568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:03.697079 16568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:03.697170 16568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:03.697257 16568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:06:03.703227 16568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:38051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:03.704823 16568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:03.706243 16568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:03.717522 16576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:03.717545 16574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:03.717536 16573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:03.717747 16568 server_base.cc:1034] running on GCE node
I20260929 03:06:03.718344 16568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:03.719031 16568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:03.720185 16568 hybrid_clock.cc:648] HybridClock initialized: now 1790651163720162 us; error 47 us; skew 500 ppm
Sep 29 03:06:03 dist-test-slave-bdps krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651163, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:06:03.723174 16568 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:06:03.724118 16568 webserver.cc:460] Webserver started at http://127.7.209.67:43997/ using document root <none> and password file <none>
I20260929 03:06:03.724732 16568 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:03.724818 16568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:03.725049 16568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:03.727499 16568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "5cfb084c06a34e6bb6e7178610824072"
format_stamp: "Formatted at 2026-09-29 03:06:03 on dist-test-slave-bdps"
server_key: "f6baddd68d8a2656035c27a7f08de6de"
server_key_iv: "3cc2fb77808f5d18a8393aa70c605060"
server_key_version: "encryptionkey@0"
I20260929 03:06:03.728139 16568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "5cfb084c06a34e6bb6e7178610824072"
format_stamp: "Formatted at 2026-09-29 03:06:03 on dist-test-slave-bdps"
server_key: "f6baddd68d8a2656035c27a7f08de6de"
server_key_iv: "3cc2fb77808f5d18a8393aa70c605060"
server_key_version: "encryptionkey@0"
I20260929 03:06:03.732743 16568 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 03:06:03.735610 16583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:03.737030 16568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:06:03.737135 16568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "5cfb084c06a34e6bb6e7178610824072"
format_stamp: "Formatted at 2026-09-29 03:06:03 on dist-test-slave-bdps"
server_key: "f6baddd68d8a2656035c27a7f08de6de"
server_key_iv: "3cc2fb77808f5d18a8393aa70c605060"
server_key_version: "encryptionkey@0"
I20260929 03:06:03.737247 16568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:03.766350 16568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:03.770349 16568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:03.770578 16568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:03.771361 16568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:03.772374 16568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:03.772423 16568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:03.772469 16568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:03.772498 16568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:03.782330 16568 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:33295
I20260929 03:06:03.782366 16696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:33295 every 8 connection(s)
I20260929 03:06:03.783383 16568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:06:03.786015 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 16568
I20260929 03:06:03.786183 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:06:03.786574 8005 external_mini_cluster.cc:1428] Setting key dc90f7fca7a00c7c29760d8ddaa7ccf4
Sep 29 03:06:03 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651163, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:06:03.797923 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:03.785561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:40083 (local address 127.7.209.126:38051)
0929 03:06:03.785759 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:03.785763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:03.786897 (+ 1134us) server_negotiation.cc:407] Connection header received
0929 03:06:03.788500 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:03.788503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:03.788565 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:03.788642 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:03.790016 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:03.790593 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:03.791826 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:03.792054 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:03.792144 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:03.794693 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:03.794696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:03.794699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:06:03.794726 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:03.796120 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:03.796569 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:03.796571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:03.796573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:03.796651 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:03.796909 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:03.796911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:03.796913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:03.797096 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:03.797181 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:03.797661 (+ 480us) server_negotiation.cc:299] Negotiation successful
0929 03:06:03.797777 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260929 03:06:03.798895 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:03.786135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:40083)
0929 03:06:03.786728 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:03.786781 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:03.788280 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:03.788852 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:03.788860 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:03.789258 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:03.789871 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:03.789886 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:03.790714 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:03.790717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:03.791704 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:03.791713 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:03.792187 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:03.792192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:03.792381 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:03.792983 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:03.793001 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:03.794534 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:06:03.796230 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:06:03.796234 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:06:03.796247 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:06:03.796491 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:06:03.796714 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:06:03.796716 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:06:03.796717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:06:03.796842 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:06:03.797166 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:06:03.797171 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:06:03.797423 (+ 252us) client_negotiation.cc:769] Sending connection context
0929 03:06:03.797579 (+ 156us) client_negotiation.cc:240] Negotiation successful
0929 03:06:03.797913 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":148,"threads_started":1}
I20260929 03:06:03.800230 16697 heartbeater.cc:347] Connected to a master server at 127.7.209.126:38051
I20260929 03:06:03.800503 16697 heartbeater.cc:464] Registering TS with master...
I20260929 03:06:03.801110 16697 heartbeater.cc:511] Master 127.7.209.126:38051 requested a full tablet report, sending...
I20260929 03:06:03.802253 16237 ts_manager.cc:194] Registered new tserver with Master: 5cfb084c06a34e6bb6e7178610824072 (127.7.209.67:33295)
I20260929 03:06:03.802876 16237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:40083
I20260929 03:06:03.812922 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab.
Entry for principal oryx/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab.
WARNING: no policy specified for oryx/127.7.209.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab.
Entry for principal oryx/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab.
WARNING: no policy specified for oryx/127.7.209.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.209.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab.
Entry for principal oryx/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab.
I20260929 03:06:03.887766 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 16300
I20260929 03:06:03.894076 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 16434
I20260929 03:06:03.899255 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 16568
I20260929 03:06:03.904017 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 16206
I20260929 03:06:03.910148 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:38051
--webserver_interface=127.7.209.126
--webserver_port=32771
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:38051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:06:04.035403 16706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:04.035686 16706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:04.035725 16706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:04.039687 16706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:06:04.039739 16706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:04.039757 16706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:04.039783 16706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:06:04.039812 16706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:06:04.044921 16706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:38051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:38051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=32771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:04.046123 16706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:04.047391 16706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:04.057205 16712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:04.057229 16714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:04.057209 16711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:04.057585 16706 server_base.cc:1034] running on GCE node
I20260929 03:06:04.058308 16706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:04.059706 16706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:04.060889 16706 hybrid_clock.cc:648] HybridClock initialized: now 1790651164060868 us; error 37 us; skew 500 ppm
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651164, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:06:04.064041 16706 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:06:04.065099 16706 webserver.cc:460] Webserver started at http://127.7.209.126:32771/ using document root <none> and password file <none>
I20260929 03:06:04.065785 16706 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:04.065872 16706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:04.070464 16706 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:06:04.072791 16721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:04.074242 16706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:06:04.074360 16706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "21469e7427f54891b23e0e9cd6774fa3"
format_stamp: "Formatted at 2026-09-29 03:06:02 on dist-test-slave-bdps"
server_key: "4e1c15c7a722c72720ae11f4f8d0704a"
server_key_iv: "a646f02095732103b8b7bc3535aae960"
server_key_version: "encryptionkey@0"
I20260929 03:06:04.074982 16706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:04.115283 16706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:04.118821 16706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:04.119036 16706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:04.127435 16706 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:38051
I20260929 03:06:04.127445 16773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:38051 every 8 connection(s)
I20260929 03:06:04.128891 16706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:06:04.134564 16774 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:06:04.136561 16774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: Bootstrap starting.
I20260929 03:06:04.137986 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 16706
I20260929 03:06:04.138501 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:38557
--local_ip_for_outbound_sockets=127.7.209.65
--tserver_master_addrs=127.7.209.126:38051
--webserver_port=46731
--webserver_interface=127.7.209.65
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.209.65
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:06:04.160941 16774 log.cc:824] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: Log is configured to *not* fsync() on all Append() calls
I20260929 03:06:04.171077 16774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:06:04.171584 16774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: Bootstrap complete.
I20260929 03:06:04.177842 16774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21469e7427f54891b23e0e9cd6774fa3" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38051 } }
I20260929 03:06:04.178436 16774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21469e7427f54891b23e0e9cd6774fa3, State: Initialized, Role: FOLLOWER
I20260929 03:06:04.179144 16774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21469e7427f54891b23e0e9cd6774fa3" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38051 } }
I20260929 03:06:04.179311 16774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:06:04.179380 16774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:06:04.179482 16774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 1 FOLLOWER]: Advancing to term 2
I20260929 03:06:04.182359 16774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21469e7427f54891b23e0e9cd6774fa3" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38051 } }
I20260929 03:06:04.182811 16774 leader_election.cc:302] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21469e7427f54891b23e0e9cd6774fa3; no voters:
I20260929 03:06:04.183241 16774 leader_election.cc:288] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 03:06:04.183369 16778 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 03:06:04.183641 16778 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [term 2 LEADER]: Becoming Leader. State: Replica: 21469e7427f54891b23e0e9cd6774fa3, State: Running, Role: LEADER
I20260929 03:06:04.184006 16778 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21469e7427f54891b23e0e9cd6774fa3" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38051 } }
I20260929 03:06:04.184757 16774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:06:04.185709 16780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21469e7427f54891b23e0e9cd6774fa3. Latest consensus state: current_term: 2 leader_uuid: "21469e7427f54891b23e0e9cd6774fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21469e7427f54891b23e0e9cd6774fa3" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38051 } } }
I20260929 03:06:04.185849 16780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [sys.catalog]: This master's current role is: LEADER
I20260929 03:06:04.185746 16779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "21469e7427f54891b23e0e9cd6774fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21469e7427f54891b23e0e9cd6774fa3" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38051 } } }
I20260929 03:06:04.186121 16779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3 [sys.catalog]: This master's current role is: LEADER
I20260929 03:06:04.191186 16793 catalog_manager.cc:1259] Loaded cluster ID: 40cc8917791d49e4865845269497c507
I20260929 03:06:04.191249 16793 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: loading cluster ID for follower catalog manager: success
I20260929 03:06:04.193846 16793 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: acquiring CA information for follower catalog manager: success
I20260929 03:06:04.195293 16793 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 03:06:04.195421 16787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:06:04.196339 16787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:06:04.196705 16787 catalog_manager.cc:1259] Loaded cluster ID: 40cc8917791d49e4865845269497c507
I20260929 03:06:04.196753 16787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:06:04.197285 16787 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:06:04.197713 16787 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 21469e7427f54891b23e0e9cd6774fa3: Loaded TSK: 0
I20260929 03:06:04.198315 16787 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 03:06:04.308583 16776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:04.308905 16776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:04.308975 16776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:04.313261 16776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:04.313336 16776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:04.313422 16776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:06:04.318993 16776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.65.keytab
--principal=oryx/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:38557
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=46731
--tserver_master_addrs=127.7.209.126:38051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:04.320235 16776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:04.321456 16776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:04.334201 16803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:04.334193 16801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:04.334432 16776 server_base.cc:1034] running on GCE node
W20260929 03:06:04.337004 16800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:04.337558 16776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:04.338539 16776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:04.339740 16776 hybrid_clock.cc:648] HybridClock initialized: now 1790651164339705 us; error 49 us; skew 500 ppm
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651164, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:06:04.343393 16776 init.cc:377] Logged in from keytab as oryx/127.7.209.65@KRBTEST.COM (short username oryx)
I20260929 03:06:04.344431 16776 webserver.cc:460] Webserver started at http://127.7.209.65:46731/ using document root <none> and password file <none>
I20260929 03:06:04.345072 16776 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:04.345157 16776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:04.349783 16776 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:06:04.352219 16810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:04.353749 16776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:06:04.353863 16776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "6008309c90e74d8186634e11c37c1670"
format_stamp: "Formatted at 2026-09-29 03:06:03 on dist-test-slave-bdps"
server_key: "5cefe7a502366bd9ac56f95afa813231"
server_key_iv: "7d24b6e0da67316182342391cb96843a"
server_key_version: "encryptionkey@0"
I20260929 03:06:04.354475 16776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:04.366267 16776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:04.370507 16776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:04.370759 16776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:04.371508 16776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:04.372509 16776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:04.372557 16776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:04.372613 16776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:04.372642 16776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:04.382344 16776 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:38557
I20260929 03:06:04.382390 16923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:38557 every 8 connection(s)
I20260929 03:06:04.383558 16776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:06:04.387877 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 16776
I20260929 03:06:04.388353 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:43559
--local_ip_for_outbound_sockets=127.7.209.66
--tserver_master_addrs=127.7.209.126:38051
--webserver_port=43917
--webserver_interface=127.7.209.66
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.209.66
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 03:06:04.398756 16926 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:04.386079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:49829 (local address 127.7.209.126:38051)
0929 03:06:04.386623 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:04.386642 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:04.386979 (+ 337us) server_negotiation.cc:407] Connection header received
0929 03:06:04.388739 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:04.388753 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:04.389118 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:04.389491 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:04.391115 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.392211 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.393454 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.394012 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.394187 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.397956 (+ 3769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:49829: BlockingRecv error: recv got EOF from 127.7.209.65:49829 (error 108)
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":121,"threads_started":1}
I20260929 03:06:04.398804 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:04.386255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:49829)
0929 03:06:04.386783 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:04.386840 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:04.388260 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:04.389598 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:04.389632 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:04.390176 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:04.390916 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.390937 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.392359 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.392362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.393329 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.393338 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.394179 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.394183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.394375 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.394976 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:04.394993 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:04.397749 (+ 2756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":143,"threads_started":1}
W20260929 03:06:04.399273 16924 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:38051 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.209.126:38051: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:04.404518 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:04.399762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:52409)
0929 03:06:04.399879 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:04.399894 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:04.399970 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:04.400185 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:04.400188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:04.400342 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:04.400435 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.400440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.401143 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.401146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.401744 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.401752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.402163 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.402167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.402253 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.402730 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:04.402741 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:04.404392 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 03:06:04.404650 16926 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:04.399816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:52409 (local address 127.7.209.126:38051)
0929 03:06:04.399914 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:04.399917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:04.399931 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:06:04.400052 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:04.400054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:04.400098 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:04.400171 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:04.400535 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.401047 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.401855 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.402072 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.402156 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.404538 (+ 2382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:52409: BlockingRecv error: recv got EOF from 127.7.209.65:52409 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:04.409859 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:04.405047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:35179)
0929 03:06:04.405138 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:04.405152 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:04.405225 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:04.405449 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:04.405451 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:04.405584 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:04.405699 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.405704 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.406398 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.406401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.406931 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.406938 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.407349 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.407352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.407422 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.407899 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:04.407913 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:04.409709 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260929 03:06:04.410063 16926 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:04.405145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:35179 (local address 127.7.209.126:38051)
0929 03:06:04.405268 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:04.405272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:04.405284 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:06:04.405317 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:04.405320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:04.405361 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:04.405424 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:04.405796 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.406303 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.407031 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.407258 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.407342 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.409837 (+ 2495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:35179: BlockingRecv error: recv got EOF from 127.7.209.65:35179 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260929 03:06:04.410097 16924 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 03:06:04.518798 16928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:04.519104 16928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:04.519167 16928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:04.523203 16928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:04.523268 16928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:04.523344 16928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:06:04.528690 16928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.66.keytab
--principal=oryx/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:43559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=43917
--tserver_master_addrs=127.7.209.126:38051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:04.530078 16928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:04.531337 16928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:04.541875 16933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:04.541879 16936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:04.541915 16934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:04.542135 16928 server_base.cc:1034] running on GCE node
I20260929 03:06:04.542629 16928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:04.543296 16928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:04.544447 16928 hybrid_clock.cc:648] HybridClock initialized: now 1790651164544429 us; error 32 us; skew 500 ppm
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651164, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:06:04.547379 16928 init.cc:377] Logged in from keytab as oryx/127.7.209.66@KRBTEST.COM (short username oryx)
I20260929 03:06:04.548300 16928 webserver.cc:460] Webserver started at http://127.7.209.66:43917/ using document root <none> and password file <none>
I20260929 03:06:04.548892 16928 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:04.548975 16928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:04.553278 16928 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:06:04.555554 16943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:04.556903 16928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 03:06:04.557008 16928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "6eb87f3418a2427db4285b623a9925c2"
format_stamp: "Formatted at 2026-09-29 03:06:03 on dist-test-slave-bdps"
server_key: "57d6b8d4d95616e7358c0666ac982ef5"
server_key_iv: "64d8cf47eb7eff5960ebe78bc86d4e11"
server_key_version: "encryptionkey@0"
I20260929 03:06:04.557643 16928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:04.581127 16928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:04.585170 16928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:04.585403 16928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:04.586215 16928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:04.587321 16928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:04.587375 16928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:04.587435 16928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:04.587459 16928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:04.597906 16928 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:43559
I20260929 03:06:04.597925 17056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:43559 every 8 connection(s)
I20260929 03:06:04.599337 16928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:06:04.607766 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 16928
I20260929 03:06:04.608385 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:33295
--local_ip_for_outbound_sockets=127.7.209.67
--tserver_master_addrs=127.7.209.126:38051
--webserver_port=43997
--webserver_interface=127.7.209.67
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.209.67
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 03:06:04.612262 16926 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:04.601622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:54747 (local address 127.7.209.126:38051)
0929 03:06:04.601791 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:04.601797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:04.602908 (+ 1111us) server_negotiation.cc:407] Connection header received
0929 03:06:04.604323 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:04.604326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:04.604388 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:04.604470 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:04.605916 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.606507 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.607765 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.608104 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.608228 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.612093 (+ 3865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:54747: BlockingRecv error: recv got EOF from 127.7.209.66:54747 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:06:04.612967 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:04.602187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:54747)
0929 03:06:04.602721 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:04.602775 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:04.604099 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:04.604658 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:04.604666 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:04.605095 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:04.605781 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.605801 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.606603 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.606605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.607639 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.607649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.608313 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.608317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.608525 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.609264 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:04.609282 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:04.611902 (+ 2620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":121,"threads_started":1}
W20260929 03:06:04.613566 17057 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:38051 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.209.126:38051: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:04.619678 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:04.614089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:45527)
0929 03:06:04.614255 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:04.614271 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:04.614362 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:04.614614 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:04.614617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:04.614793 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:04.614923 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.614931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.615641 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.615644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.616514 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.616525 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.616975 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.616979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.617071 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.617578 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:04.617593 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:04.619536 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":79}
W20260929 03:06:04.619787 16926 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:04.614232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:45527 (local address 127.7.209.126:38051)
0929 03:06:04.614346 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:04.614350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:04.614362 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:06:04.614458 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:04.614460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:04.614512 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:04.614578 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:04.615023 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.615546 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.616651 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.616870 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.616948 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.619670 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:45527: BlockingRecv error: recv got EOF from 127.7.209.66:45527 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:04.625141 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:04.620277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:52095)
0929 03:06:04.620742 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:04.620755 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:04.620838 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:04.621039 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:04.621042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:04.621193 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:04.621262 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.621267 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.622068 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.622071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.622595 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.622603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.623003 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.623007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.623077 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.623534 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:04.623545 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:04.625012 (+ 1467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":123,"threads_started":1}
W20260929 03:06:04.625258 16926 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:04.620338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:52095 (local address 127.7.209.126:38051)
0929 03:06:04.620448 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:04.620452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:04.620817 (+ 365us) server_negotiation.cc:407] Connection header received
0929 03:06:04.620910 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:04.620913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:04.620954 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:04.621018 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:04.621362 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.621972 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.622697 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.622914 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.622994 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.625144 (+ 2150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:52095: BlockingRecv error: recv got EOF from 127.7.209.66:52095 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260929 03:06:04.625504 17057 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 03:06:04.738529 17060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:04.738857 17060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:04.738925 17060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:04.743078 17060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:04.743134 17060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:04.743211 17060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:06:04.748634 17060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/krb5kdc/oryx_127.7.209.67.keytab
--principal=oryx/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:33295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=43997
--tserver_master_addrs=127.7.209.126:38051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:04.749868 17060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:04.751068 17060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:04.761758 17066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:04.761890 17060 server_base.cc:1034] running on GCE node
W20260929 03:06:04.761761 17067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:04.761758 17069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:04.762575 17060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:04.763264 17060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:04.764424 17060 hybrid_clock.cc:648] HybridClock initialized: now 1790651164764405 us; error 38 us; skew 500 ppm
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651164, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:06:04.767444 17060 init.cc:377] Logged in from keytab as oryx/127.7.209.67@KRBTEST.COM (short username oryx)
I20260929 03:06:04.768440 17060 webserver.cc:460] Webserver started at http://127.7.209.67:43997/ using document root <none> and password file <none>
I20260929 03:06:04.769101 17060 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:04.769186 17060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:04.773922 17060 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260929 03:06:04.776401 17076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:04.778126 17060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:06:04.778244 17060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "5cfb084c06a34e6bb6e7178610824072"
format_stamp: "Formatted at 2026-09-29 03:06:03 on dist-test-slave-bdps"
server_key: "f6baddd68d8a2656035c27a7f08de6de"
server_key_iv: "3cc2fb77808f5d18a8393aa70c605060"
server_key_version: "encryptionkey@0"
I20260929 03:06:04.779011 17060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:04.798091 17060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:04.801493 17060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:04.801710 17060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:04.802371 17060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:04.803289 17060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:04.803339 17060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:04.803385 17060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:04.803413 17060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:04.813300 17060 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:33295
I20260929 03:06:04.813321 17189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:33295 every 8 connection(s)
I20260929 03:06:04.814558 17060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:06:04.816740 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 17060
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 03:06:04.827286 16926 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:04.816699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:34487 (local address 127.7.209.126:38051)
0929 03:06:04.816885 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:04.816891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:04.817982 (+ 1091us) server_negotiation.cc:407] Connection header received
0929 03:06:04.819409 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:04.819412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:04.819459 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:04.819548 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:04.821070 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.821836 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.823264 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.823468 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.823552 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.827173 (+ 3621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:34487: BlockingRecv error: recv got EOF from 127.7.209.67:34487 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 03:06:04.828015 17194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:04.817934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46078 (local address 127.7.209.126:38051)
0929 03:06:04.818174 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:04.818179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:04.818510 (+ 331us) server_negotiation.cc:407] Connection header received
0929 03:06:04.818683 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:04.818686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:04.818736 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:04.818825 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:04.819706 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.820552 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.821760 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.822042 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.822147 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.823651 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:04.823657 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:04.823670 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:06:04.823698 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:04.825369 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:04.825902 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:04.825908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:04.825915 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:04.826079 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:04.826349 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:04.826351 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:04.826353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:04.826732 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:04.826900 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:04.827702 (+ 802us) server_negotiation.cc:299] Negotiation successful
0929 03:06:04.827903 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
I20260929 03:06:04.828027 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:04.817193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:34487)
0929 03:06:04.817797 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:04.817862 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:04.819210 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:04.819786 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:04.819796 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:04.820297 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:04.820947 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.820963 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.821941 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.821944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.823128 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.823138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.823604 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.823607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.823788 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.824364 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:04.824383 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:04.827098 (+ 2715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":149,"threads_started":1}
W20260929 03:06:04.828642 17190 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:38051 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.209.126:38051: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:04.833729 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:04.829108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:48773)
0929 03:06:04.829197 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:04.829209 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:04.829279 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:04.829484 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:04.829487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:04.829684 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:04.829775 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.829781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.830558 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.830561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.831102 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.831110 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.831518 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.831522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.831619 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.831955 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:04.831965 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:04.833579 (+ 1614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260929 03:06:04.833827 17194 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:04.829181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:48773 (local address 127.7.209.126:38051)
0929 03:06:04.829276 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:04.829280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:04.829292 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:06:04.829362 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:04.829365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:04.829401 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:04.829458 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:04.829877 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.830459 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.831207 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.831425 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.831507 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.833730 (+ 2223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:48773: BlockingRecv error: recv got EOF from 127.7.209.67:48773 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:04 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:04.838253 17192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:04.834217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:58109)
0929 03:06:04.834306 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:04.834321 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:04.834389 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:04.834559 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:04.834561 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:04.834688 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:04.834752 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.834756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.835398 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.835400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.835922 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:04.835929 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.836316 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.836318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:04.836374 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.836705 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:04.836715 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:04.838171 (+ 1456us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260929 03:06:04.838402 17194 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:04.834280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:58109 (local address 127.7.209.126:38051)
0929 03:06:04.834413 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:04.834416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:04.834427 (+ 11us) server_negotiation.cc:407] Connection header received
0929 03:06:04.834452 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:04.834454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:04.834482 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:04.834542 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:04.834832 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.835307 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.836009 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:04.836218 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:04.836290 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:04.838294 (+ 2004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:58109: BlockingRecv error: recv got EOF from 127.7.209.67:58109 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260929 03:06:04.838415 17190 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 03:06:05 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:05 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:05.416877 17195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:05.410902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:39155)
0929 03:06:05.411250 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:05.411267 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:05.411401 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:05.411765 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:05.411768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:05.412042 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:05.412170 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:05.412178 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:05.413005 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:05.413009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:05.413674 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:05.413684 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:05.414139 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:05.414143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:05.414230 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:05.414811 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:05.414824 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:05.416732 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":107,"threads_started":1}
W20260929 03:06:05.417008 17196 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:05.411011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:39155 (local address 127.7.209.126:38051)
0929 03:06:05.411310 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:05.411315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:05.411332 (+ 17us) server_negotiation.cc:407] Connection header received
0929 03:06:05.411524 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:05.411527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:05.411592 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:05.411698 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:05.412298 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:05.412891 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:05.413793 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:05.414032 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:05.414122 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:05.416880 (+ 2758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:39155: BlockingRecv error: recv got EOF from 127.7.209.65:39155 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":87,"threads_started":1}
Sep 29 03:06:05 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:05 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:05.631872 17197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:05.626188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:32889)
0929 03:06:05.626486 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:05.626501 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:05.626631 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:05.626900 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:05.626902 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:05.627119 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:05.627241 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:05.627248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:05.628059 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:05.628061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:05.628694 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:05.628702 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:05.629156 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:05.629159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:05.629232 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:05.629829 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:05.629842 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:05.631707 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":91,"threads_started":1}
W20260929 03:06:05.631954 17196 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:05.626249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:32889 (local address 127.7.209.126:38051)
0929 03:06:05.626393 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:05.626397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:05.626609 (+ 212us) server_negotiation.cc:407] Connection header received
0929 03:06:05.626724 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:05.626726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:05.626782 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:05.626845 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:05.627355 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:05.627960 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:05.628812 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:05.629060 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:05.629142 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:05.631849 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:32889: BlockingRecv error: recv got EOF from 127.7.209.66:32889 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 03:06:05 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:05 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:05.844843 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:05.839075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:36635)
0929 03:06:05.839356 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:05.839372 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:05.839497 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:05.839755 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:05.839758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:05.839971 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:05.840088 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:05.840094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:05.840839 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:05.840841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:05.841538 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:05.841554 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:05.842109 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:05.842114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:05.842195 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:05.842781 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:05.842793 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:05.844696 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":83,"threads_started":1}
W20260929 03:06:05.844964 17196 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:05.839170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:36635 (local address 127.7.209.126:38051)
0929 03:06:05.839309 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:05.839313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:05.839468 (+ 155us) server_negotiation.cc:407] Connection header received
0929 03:06:05.839584 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:05.839586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:05.839644 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:05.839718 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:05.840190 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:05.840753 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:05.841715 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:05.841986 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:05.842067 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:05.844838 (+ 2771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:36635: BlockingRecv error: recv got EOF from 127.7.209.67:36635 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 29 03:06:06 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:06 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:06.424923 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:06.417978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:36917)
0929 03:06:06.418321 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:06.418338 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:06.418590 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:06.418920 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:06.418923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:06.419236 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:06.419412 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:06.419440 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:06.420352 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:06.420356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:06.421067 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:06.421077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:06.421747 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:06.421751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:06.421851 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:06.422450 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:06.422463 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:06.424704 (+ 2241us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":103,"threads_started":1}
W20260929 03:06:06.424994 17200 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:06.418003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:36917 (local address 127.7.209.126:38051)
0929 03:06:06.418322 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:06.418327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:06.418474 (+ 147us) server_negotiation.cc:407] Connection header received
0929 03:06:06.418727 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:06.418731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:06.418795 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:06.418879 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:06.419563 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:06.420239 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:06.421201 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:06.421549 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:06.421747 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:06.424873 (+ 3126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:36917: BlockingRecv error: recv got EOF from 127.7.209.65:36917 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":106,"threads_started":1}
Sep 29 03:06:06 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:06 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:06.638777 17201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:06.632719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:51995)
0929 03:06:06.633147 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:06.633168 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:06.633297 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:06.633551 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:06.633554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:06.633833 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:06.633955 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:06.633962 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:06.634843 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:06.634846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:06.635439 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:06.635447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:06.635937 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:06.635941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:06.636096 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:06.636657 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:06.636671 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:06.638602 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":173,"threads_started":1}
W20260929 03:06:06.638890 17200 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:06.632798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:51995 (local address 127.7.209.126:38051)
0929 03:06:06.632978 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:06.632982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:06.633245 (+ 263us) server_negotiation.cc:407] Connection header received
0929 03:06:06.633387 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:06.633390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:06.633444 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:06.633514 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:06.634069 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:06.634730 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:06.635548 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:06.635817 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:06.635928 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:06.638775 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:51995: BlockingRecv error: recv got EOF from 127.7.209.66:51995 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 29 03:06:06 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:06 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:06.863644 17202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:06.845839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:57947)
0929 03:06:06.846237 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:06.846254 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:06.846371 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:06.846608 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:06.846610 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:06.846839 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:06.847035 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:06.847042 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:06.847831 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:06.847833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:06.848471 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:06.848481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:06.848943 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:06.848946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:06.849032 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:06.861120 (+ 12088us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:06.861135 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:06.863436 (+ 2301us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":108,"threads_started":1}
W20260929 03:06:06.863713 17200 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:06.846000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:57947 (local address 127.7.209.126:38051)
0929 03:06:06.846161 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:06.846166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:06.846324 (+ 158us) server_negotiation.cc:407] Connection header received
0929 03:06:06.846450 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:06.846452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:06.846506 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:06.846581 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:06.847139 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:06.847732 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:06.848580 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:06.848836 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:06.848920 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:06.863595 (+ 14675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:57947: BlockingRecv error: recv got EOF from 127.7.209.67:57947 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 29 03:06:07 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:07 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:07.432341 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:07.426297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:44655)
0929 03:06:07.426654 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:07.426669 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:07.426887 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:07.427154 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:07.427157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:07.427419 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:07.427577 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:07.427641 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:07.428412 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:07.428415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:07.429125 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:07.429135 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:07.429529 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:07.429532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:07.429662 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:07.430261 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:07.430274 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:07.432204 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":104,"threads_started":1}
W20260929 03:06:07.432475 17203 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:07.426310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:44655 (local address 127.7.209.126:38051)
0929 03:06:07.426626 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:07.426631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:07.426779 (+ 148us) server_negotiation.cc:407] Connection header received
0929 03:06:07.426963 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:07.426965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:07.427021 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:07.427103 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:07.427737 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:07.428314 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:07.429215 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:07.429441 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:07.429507 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:07.432332 (+ 2825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:44655: BlockingRecv error: recv got EOF from 127.7.209.65:44655 (error 108)
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":104,"threads_started":1}
Sep 29 03:06:07 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:07 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:07.645726 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:07.639747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:34625)
0929 03:06:07.640033 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:07.640049 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:07.640171 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:07.640432 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:07.640434 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:07.640658 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:07.640792 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:07.640798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:07.641576 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:07.641578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:07.642186 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:07.642193 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:07.642613 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:07.642615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:07.642684 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:07.643288 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:07.643303 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:07.645521 (+ 2218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":101,"threads_started":1}
W20260929 03:06:07.645856 17203 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:07.639811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:34625 (local address 127.7.209.126:38051)
0929 03:06:07.640022 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:07.640027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:07.640125 (+ 98us) server_negotiation.cc:407] Connection header received
0929 03:06:07.640270 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:07.640273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:07.640327 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:07.640395 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:07.640905 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:07.641492 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:07.642284 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:07.642523 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:07.642621 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:07.645724 (+ 3103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:34625: BlockingRecv error: recv got EOF from 127.7.209.66:34625 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Sep 29 03:06:07 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:07 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:07.869793 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:07.864473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:34039)
0929 03:06:07.864786 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:07.864801 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:07.864913 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:07.865123 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:07.865125 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:07.865335 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:07.865448 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:07.865453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:07.866233 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:07.866235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:07.866818 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:07.866825 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:07.867203 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:07.867206 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:07.867268 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:07.867794 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:07.867805 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:07.869652 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":119,"threads_started":1}
W20260929 03:06:07.869900 17203 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:07.864526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:34039 (local address 127.7.209.126:38051)
0929 03:06:07.864707 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:07.864713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:07.864866 (+ 153us) server_negotiation.cc:407] Connection header received
0929 03:06:07.864986 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:07.864988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:07.865042 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:07.865110 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:07.865527 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:07.866151 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:07.866901 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:07.867122 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:07.867191 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:07.869794 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:34039: BlockingRecv error: recv got EOF from 127.7.209.67:34039 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 29 03:06:08 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:08 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:08.439024 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:08.433169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:35703)
0929 03:06:08.433505 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:08.433519 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:08.433657 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:08.433951 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:08.433953 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:08.434175 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:08.434293 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:08.434300 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:08.435110 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:08.435114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:08.435710 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:08.435719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:08.436143 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:08.436147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:08.436216 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:08.436809 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:08.436821 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:08.438890 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":121,"threads_started":1}
W20260929 03:06:08.439159 17208 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:08.433203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:35703 (local address 127.7.209.126:38051)
0929 03:06:08.433502 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:08.433506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:08.433587 (+ 81us) server_negotiation.cc:407] Connection header received
0929 03:06:08.433753 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:08.433756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:08.433814 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:08.433920 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:08.434390 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:08.434981 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:08.435804 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:08.436026 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:08.436095 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:08.439017 (+ 2922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:35703: BlockingRecv error: recv got EOF from 127.7.209.65:35703 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":94,"threads_started":1}
Sep 29 03:06:08 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:08 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:08.652887 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:08.646701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:45053)
0929 03:06:08.647020 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:08.647036 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:08.647164 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:08.647410 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:08.647412 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:08.647646 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:08.647781 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:08.647787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:08.648597 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:08.648599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:08.649207 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:08.649214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:08.649773 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:08.649777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:08.649873 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:08.650562 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:08.650575 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:08.652733 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":127,"threads_started":1}
W20260929 03:06:08.653025 17208 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:08.646780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:45053 (local address 127.7.209.126:38051)
0929 03:06:08.646980 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:08.646985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:08.647119 (+ 134us) server_negotiation.cc:407] Connection header received
0929 03:06:08.647250 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:08.647252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:08.647305 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:08.647379 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:08.647923 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:08.648511 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:08.649328 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:08.649655 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:08.649765 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:08.652891 (+ 3126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:45053: BlockingRecv error: recv got EOF from 127.7.209.66:45053 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Sep 29 03:06:08 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:08 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:08.876461 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:08.870683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:48195)
0929 03:06:08.871041 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:08.871057 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:08.871191 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:08.871477 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:08.871480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:08.871720 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:08.871837 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:08.871843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:08.872629 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:08.872634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:08.873243 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:08.873251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:08.873712 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:08.873716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:08.873847 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:08.874394 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:08.874405 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:08.876323 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":158,"threads_started":1}
W20260929 03:06:08.876586 17208 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:08.870703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:48195 (local address 127.7.209.126:38051)
0929 03:06:08.870854 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:08.870860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:08.871134 (+ 274us) server_negotiation.cc:407] Connection header received
0929 03:06:08.871308 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:08.871311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:08.871371 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:08.871438 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:08.871951 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:08.872525 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:08.873357 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:08.873594 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:08.873710 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:08.876461 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:48195: BlockingRecv error: recv got EOF from 127.7.209.67:48195 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 29 03:06:09 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:09 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:09.445847 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:09.439993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:47909)
0929 03:06:09.440354 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:09.440374 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:09.440483 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:09.440778 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:09.440781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:09.441000 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:09.441181 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:09.441190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:09.442053 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:09.442055 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:09.442652 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:09.442660 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:09.443089 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:09.443091 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:09.443155 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:09.443687 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:09.443698 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:09.445675 (+ 1977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":94,"threads_started":1}
W20260929 03:06:09.445956 17212 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:09.440051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:47909 (local address 127.7.209.126:38051)
0929 03:06:09.440454 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:09.440458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:09.440473 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:06:09.440568 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:09.440570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:09.440637 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:09.440726 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:09.441295 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:09.441944 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:09.442766 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:09.443000 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:09.443079 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:09.445825 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:47909: BlockingRecv error: recv got EOF from 127.7.209.65:47909 (error 108)
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":199,"threads_started":1}
Sep 29 03:06:09 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:09 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:09.659677 17213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:09.653892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:44207)
0929 03:06:09.654202 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:09.654220 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:09.654372 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:09.654690 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:09.654694 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:09.654937 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:09.655066 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:09.655074 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:09.655826 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:09.655829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:09.656420 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:09.656428 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:09.656832 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:09.656835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:09.656910 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:09.657490 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:09.657503 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:09.659537 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":101,"threads_started":1}
W20260929 03:06:09.659794 17212 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:09.653989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:44207 (local address 127.7.209.126:38051)
0929 03:06:09.654135 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:09.654140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:09.654296 (+ 156us) server_negotiation.cc:407] Connection header received
0929 03:06:09.654481 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:09.654484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:09.654542 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:09.654627 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:09.655169 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:09.655743 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:09.656501 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:09.656729 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:09.656797 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:09.659685 (+ 2888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:44207: BlockingRecv error: recv got EOF from 127.7.209.66:44207 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 29 03:06:09 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:09 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:09.883224 17214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:09.877274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:44273)
0929 03:06:09.877627 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:09.877646 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:09.877781 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:09.878017 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:09.878020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:09.878259 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:09.878381 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:09.878410 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:09.879271 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:09.879275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:09.879960 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:09.879972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:09.880388 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:09.880391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:09.880461 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:09.881029 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:09.881041 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:09.883057 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":130,"threads_started":1}
W20260929 03:06:09.883350 17212 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:09.877332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:44273 (local address 127.7.209.126:38051)
0929 03:06:09.877523 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:09.877529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:09.877727 (+ 198us) server_negotiation.cc:407] Connection header received
0929 03:06:09.877874 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:09.877876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:09.877934 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:09.877994 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:09.878528 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:09.879148 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:09.880062 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:09.880296 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:09.880371 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:09.883219 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:44273: BlockingRecv error: recv got EOF from 127.7.209.67:44273 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 29 03:06:10 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:10 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:10.452950 17215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:10.446783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:35879)
0929 03:06:10.447100 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:10.447116 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:10.447240 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:10.447480 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:10.447483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:10.447731 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:10.447845 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:10.447851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:10.448662 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:10.448666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:10.449285 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:10.449295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:10.449738 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:10.449741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:10.449808 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:10.450382 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:10.450393 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:10.452774 (+ 2381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":91,"threads_started":1}
W20260929 03:06:10.453138 17216 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:10.446870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:35879 (local address 127.7.209.126:38051)
0929 03:06:10.447182 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:10.447186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:10.447200 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:06:10.447322 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:10.447324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:10.447373 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:10.447456 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:10.447937 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:10.448528 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:10.449388 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:10.449648 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:10.449730 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:10.452983 (+ 3253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:35879: BlockingRecv error: recv got EOF from 127.7.209.65:35879 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":121,"threads_started":1}
Sep 29 03:06:10 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:10 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:10.666658 17217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:10.660723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:54803)
0929 03:06:10.661032 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:10.661048 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:10.661194 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:10.661506 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:10.661509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:10.661838 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:10.661986 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:10.661995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:10.662764 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:10.662767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:10.663450 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:10.663462 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:10.663897 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:10.663901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:10.663975 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:10.664535 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:10.664547 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:10.666488 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":109,"threads_started":1}
W20260929 03:06:10.666756 17216 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:10.660771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:54803 (local address 127.7.209.126:38051)
0929 03:06:10.661098 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:10.661103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:10.661120 (+ 17us) server_negotiation.cc:407] Connection header received
0929 03:06:10.661302 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:10.661305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:10.661373 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:10.661454 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:10.662097 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:10.662665 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:10.663542 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:10.663770 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:10.663842 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:10.666632 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:54803: BlockingRecv error: recv got EOF from 127.7.209.66:54803 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 03:06:10 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:10 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:10.890202 17218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:10.884117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:55713)
0929 03:06:10.884406 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:10.884422 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:10.884542 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:10.884779 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:10.884782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:10.885067 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:10.885190 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:10.885196 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:10.886037 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:10.886040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:10.886723 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:10.886735 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:10.887149 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:10.887153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:10.887224 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:10.887865 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:10.887878 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:10.890025 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":109,"threads_started":1}
W20260929 03:06:10.890726 17216 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:10.884159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:55713 (local address 127.7.209.126:38051)
0929 03:06:10.884328 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:10.884334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:10.884493 (+ 159us) server_negotiation.cc:407] Connection header received
0929 03:06:10.884624 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:10.884626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:10.884686 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:10.884759 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:10.885299 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:10.885941 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:10.886820 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:10.887051 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:10.887131 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:10.890611 (+ 3480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:55713: BlockingRecv error: recv got EOF from 127.7.209.67:55713 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 29 03:06:11 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:11 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:11.460180 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:11.453956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:52085)
0929 03:06:11.454402 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:11.454416 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:11.454543 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:11.454840 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:11.454843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:11.455104 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:11.455255 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:11.455265 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:11.456263 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:11.456266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:11.456871 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:11.456880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:11.457275 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:11.457278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:11.457342 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:11.457960 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:11.457973 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:11.460003 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":112,"threads_started":1}
W20260929 03:06:11.460304 17220 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:11.454046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:52085 (local address 127.7.209.126:38051)
0929 03:06:11.454340 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:11.454344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:11.454489 (+ 145us) server_negotiation.cc:407] Connection header received
0929 03:06:11.454624 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:11.454626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:11.454674 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:11.454769 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:11.455375 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:11.456145 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:11.456960 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:11.457194 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:11.457265 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:11.460151 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:52085: BlockingRecv error: recv got EOF from 127.7.209.65:52085 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":98,"threads_started":1}
Sep 29 03:06:11 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:11 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
W20260929 03:06:11.676985 17220 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:11.667981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:45333 (local address 127.7.209.126:38051)
0929 03:06:11.668132 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:11.668138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:11.668310 (+ 172us) server_negotiation.cc:407] Connection header received
0929 03:06:11.668549 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:11.668552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:11.668632 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:11.668693 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:11.669507 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:11.670422 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:11.671677 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:11.672021 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:11.672104 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:11.676809 (+ 4705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:45333: BlockingRecv error: recv got EOF from 127.7.209.66:45333 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 03:06:11.677891 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:11.667505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:45333)
0929 03:06:11.668208 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:11.668228 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:11.668418 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:11.668915 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:11.668919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:11.669186 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:11.669348 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:11.669358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:11.670594 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:11.670598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:11.671526 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:11.671539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:11.672315 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:11.672318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:11.672404 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:11.673396 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:11.673414 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:11.677502 (+ 4088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":125,"threads_started":1}
Sep 29 03:06:11 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:11 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:11.900331 17222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:11.891086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:33199)
0929 03:06:11.891444 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:11.891461 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:11.891591 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:11.891830 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:11.891832 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:11.892071 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:11.892191 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:11.892197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:11.892950 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:11.892953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:11.893551 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:11.893559 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:11.894009 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:11.894012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:11.894084 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:11.897773 (+ 3689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:11.897786 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:11.900163 (+ 2377us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":102,"threads_started":1}
W20260929 03:06:11.900442 17220 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:11.891147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:33199 (local address 127.7.209.126:38051)
0929 03:06:11.891323 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:11.891328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:11.891545 (+ 217us) server_negotiation.cc:407] Connection header received
0929 03:06:11.891668 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:11.891670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:11.891728 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:11.891799 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:11.892279 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:11.892855 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:11.893686 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:11.893909 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:11.893983 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:11.900329 (+ 6346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:33199: BlockingRecv error: recv got EOF from 127.7.209.67:33199 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 29 03:06:12 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:12 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:12.467376 17223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:12.461169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:40747)
0929 03:06:12.461518 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:12.461534 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:12.461688 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:12.461986 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:12.461989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:12.462249 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:12.462366 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:12.462372 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:12.463200 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:12.463204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:12.463819 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:12.463828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:12.464297 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:12.464301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:12.464388 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:12.465041 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:12.465058 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:12.467236 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":117,"threads_started":1}
W20260929 03:06:12.467530 17224 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:12.461328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:40747 (local address 127.7.209.126:38051)
0929 03:06:12.461585 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:12.461589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:12.461621 (+ 32us) server_negotiation.cc:407] Connection header received
0929 03:06:12.461779 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:12.461781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:12.461833 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:12.461940 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:12.462492 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:12.463093 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:12.463937 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:12.464193 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:12.464282 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:12.467394 (+ 3112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:40747: BlockingRecv error: recv got EOF from 127.7.209.65:40747 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":69,"threads_started":1}
Sep 29 03:06:12 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:12 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:12.687036 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:12.679081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:39227)
0929 03:06:12.679449 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:12.679465 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:12.679604 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:12.679899 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:12.679902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:12.680181 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:12.680323 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:12.680332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:12.681738 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:12.681742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:12.682603 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:12.682613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:12.683423 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:12.683427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:12.683530 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:12.684363 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:12.684378 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:12.686849 (+ 2471us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":124,"threads_started":1}
W20260929 03:06:12.687122 17224 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:12.679107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:39227 (local address 127.7.209.126:38051)
0929 03:06:12.679289 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:12.679294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:12.679534 (+ 240us) server_negotiation.cc:407] Connection header received
0929 03:06:12.679721 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:12.679724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:12.679788 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:12.679860 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:12.680461 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:12.681566 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:12.682949 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:12.683307 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:12.683417 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:12.686988 (+ 3571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:39227: BlockingRecv error: recv got EOF from 127.7.209.66:39227 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 29 03:06:12 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:12 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:12.907287 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:12.901347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:40973)
0929 03:06:12.901728 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:12.901748 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:12.901875 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:12.902112 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:12.902114 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:12.902338 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:12.902454 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:12.902460 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:12.903393 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:12.903396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:12.904139 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:12.904149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:12.904583 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:12.904586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:12.904667 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:12.905214 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:12.905226 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:12.907116 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":132,"threads_started":1}
W20260929 03:06:12.907399 17224 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:12.901462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:40973 (local address 127.7.209.126:38051)
0929 03:06:12.901696 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:12.901700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:12.901824 (+ 124us) server_negotiation.cc:407] Connection header received
0929 03:06:12.901955 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:12.901957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:12.902012 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:12.902085 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:12.902584 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:12.903283 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:12.904246 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:12.904485 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:12.904571 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:12.907285 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:40973: BlockingRecv error: recv got EOF from 127.7.209.67:40973 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Sep 29 03:06:13 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:13 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:13.474109 17227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:13.468236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:57103)
0929 03:06:13.468576 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:13.468591 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:13.468735 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:13.469064 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:13.469067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:13.469316 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:13.469439 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:13.469446 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:13.470235 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:13.470238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:13.470826 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:13.470835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:13.471239 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:13.471242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:13.471305 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:13.471903 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:13.471916 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:13.473951 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":110,"threads_started":1}
W20260929 03:06:13.474231 17228 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:13.468303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:57103 (local address 127.7.209.126:38051)
0929 03:06:13.468579 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:13.468584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:13.468677 (+ 93us) server_negotiation.cc:407] Connection header received
0929 03:06:13.468851 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:13.468854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:13.468919 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:13.469009 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:13.469539 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:13.470146 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:13.470911 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:13.471133 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:13.471250 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:13.474085 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:57103: BlockingRecv error: recv got EOF from 127.7.209.65:57103 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":69,"threads_started":1}
Sep 29 03:06:13 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:13 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:13.694563 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:13.687859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:46251)
0929 03:06:13.688196 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:13.688210 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:13.688341 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:13.688661 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:13.688664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:13.688934 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:13.689112 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:13.689126 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:13.690059 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:13.690064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:13.690685 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:13.690698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:13.691106 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:13.691109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:13.691171 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:13.691799 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:13.691813 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:13.694408 (+ 2595us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":127,"threads_started":1}
W20260929 03:06:13.694703 17228 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:13.687925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:46251 (local address 127.7.209.126:38051)
0929 03:06:13.688069 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:13.688073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:13.688313 (+ 240us) server_negotiation.cc:407] Connection header received
0929 03:06:13.688443 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:13.688446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:13.688518 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:13.688627 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:13.689241 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:13.689941 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:13.690789 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:13.691021 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:13.691095 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:13.694579 (+ 3484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:46251: BlockingRecv error: recv got EOF from 127.7.209.66:46251 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 29 03:06:13 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:13 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:13.913905 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:13.908313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:59503)
0929 03:06:13.908625 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:13.908640 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:13.908760 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:13.908982 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:13.908984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:13.909220 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:13.909335 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:13.909341 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:13.910115 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:13.910117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:13.910710 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:13.910718 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:13.911088 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:13.911090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:13.911149 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:13.911695 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:13.911706 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:13.913740 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":112,"threads_started":1}
W20260929 03:06:13.913997 17228 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:13.908344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:59503 (local address 127.7.209.126:38051)
0929 03:06:13.908496 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:13.908501 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:13.908716 (+ 215us) server_negotiation.cc:407] Connection header received
0929 03:06:13.908831 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:13.908833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:13.908887 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:13.908953 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:13.909418 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:13.910026 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:13.910797 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:13.911011 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:13.911079 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:13.913883 (+ 2804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:59503: BlockingRecv error: recv got EOF from 127.7.209.67:59503 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 29 03:06:14 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:14 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:14.481309 17231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:14.475149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:41739)
0929 03:06:14.475524 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:14.475542 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:14.475661 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:14.476026 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:14.476030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:14.476290 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:14.476411 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:14.476418 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:14.477275 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:14.477279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:14.477914 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:14.477923 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:14.478411 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:14.478415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:14.478536 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:14.479087 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:14.479098 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:14.481110 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":102,"threads_started":1}
W20260929 03:06:14.481419 17232 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:14.475241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:41739 (local address 127.7.209.126:38051)
0929 03:06:14.475585 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:14.475591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:14.475609 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:06:14.475795 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:14.475799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:14.475863 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:14.475984 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:14.476542 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:14.477156 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:14.478032 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:14.478307 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:14.478399 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:14.481279 (+ 2880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:41739: BlockingRecv error: recv got EOF from 127.7.209.65:41739 (error 108)
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":108,"threads_started":1}
Sep 29 03:06:14 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:14 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:14.701592 17233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:14.695650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:46077)
0929 03:06:14.696008 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:14.696025 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:14.696158 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:14.696434 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:14.696438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:14.696690 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:14.696804 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:14.696811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:14.697740 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:14.697744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:14.698429 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:14.698440 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:14.698897 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:14.698901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:14.698982 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:14.699498 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:14.699509 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:14.701452 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":112,"threads_started":1}
W20260929 03:06:14.701740 17232 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:14.695739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:46077 (local address 127.7.209.126:38051)
0929 03:06:14.695946 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:14.695952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:14.696105 (+ 153us) server_negotiation.cc:407] Connection header received
0929 03:06:14.696260 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:14.696263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:14.696322 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:14.696396 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:14.696924 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:14.697588 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:14.698545 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:14.698792 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:14.698881 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:14.701586 (+ 2705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:46077: BlockingRecv error: recv got EOF from 127.7.209.66:46077 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 29 03:06:14 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:14 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:14.920639 17234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:14.914805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:46725)
0929 03:06:14.915133 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:14.915151 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:14.915302 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:14.915569 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:14.915572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:14.915820 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:14.915933 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:14.915939 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:14.916746 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:14.916750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:14.917355 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:14.917364 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:14.917888 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:14.917892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:14.917986 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:14.918579 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:14.918591 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:14.920497 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":105,"threads_started":1}
W20260929 03:06:14.920755 17232 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:14.914992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:46725 (local address 127.7.209.126:38051)
0929 03:06:14.915150 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:14.915155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:14.915246 (+ 91us) server_negotiation.cc:407] Connection header received
0929 03:06:14.915398 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:14.915401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:14.915458 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:14.915528 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:14.916054 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:14.916634 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:14.917474 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:14.917774 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:14.917869 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:14.920641 (+ 2772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:46725: BlockingRecv error: recv got EOF from 127.7.209.67:46725 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 29 03:06:15 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:15 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:15.488312 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:15.482210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:54397)
0929 03:06:15.482553 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:15.482568 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:15.482692 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:15.482956 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:15.482959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:15.483218 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:15.483345 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:15.483351 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:15.484195 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:15.484199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:15.484810 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:15.484819 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:15.485298 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:15.485302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:15.485395 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:15.486055 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:15.486071 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:15.488167 (+ 2096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":115,"threads_started":1}
W20260929 03:06:15.488473 17236 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:15.482268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:54397 (local address 127.7.209.126:38051)
0929 03:06:15.482556 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:15.482560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:15.482637 (+ 77us) server_negotiation.cc:407] Connection header received
0929 03:06:15.482796 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:15.482798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:15.482853 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:15.482932 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:15.483477 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:15.484079 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:15.484933 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:15.485189 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:15.485293 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:15.488322 (+ 3029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:54397: BlockingRecv error: recv got EOF from 127.7.209.65:54397 (error 108)
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":125,"threads_started":1}
Sep 29 03:06:15 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:15 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:15.708508 17237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:15.702490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:52783)
0929 03:06:15.702801 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:15.702815 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:15.702951 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:15.703234 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:15.703237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:15.703476 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:15.703594 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:15.703601 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:15.704414 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:15.704418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:15.705006 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:15.705014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:15.705477 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:15.705481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:15.705564 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:15.706152 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:15.706163 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:15.708348 (+ 2185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":94,"threads_started":1}
W20260929 03:06:15.708644 17236 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:15.702553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:52783 (local address 127.7.209.126:38051)
0929 03:06:15.702729 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:15.702735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:15.702892 (+ 157us) server_negotiation.cc:407] Connection header received
0929 03:06:15.703045 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:15.703048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:15.703106 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:15.703178 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:15.703720 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:15.704308 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:15.705126 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:15.705371 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:15.705462 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:15.708521 (+ 3059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:52783: BlockingRecv error: recv got EOF from 127.7.209.66:52783 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 03:06:15 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:15 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:15.927289 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:15.921458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:54819)
0929 03:06:15.921809 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:15.921824 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:15.921958 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:15.922224 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:15.922227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:15.922475 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:15.922592 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:15.922600 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:15.923408 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:15.923412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:15.924021 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:15.924030 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:15.924490 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:15.924493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:15.924585 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:15.925139 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:15.925151 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:15.927134 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":136,"threads_started":1}
W20260929 03:06:15.927412 17236 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:15.921526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:54819 (local address 127.7.209.126:38051)
0929 03:06:15.921695 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:15.921699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:15.921903 (+ 204us) server_negotiation.cc:407] Connection header received
0929 03:06:15.922053 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:15.922056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:15.922114 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:15.922190 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:15.922708 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:15.923298 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:15.924141 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:15.924383 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:15.924473 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:15.927286 (+ 2813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:54819: BlockingRecv error: recv got EOF from 127.7.209.67:54819 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 29 03:06:16 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:16 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:16.498066 17239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:16.489303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:47081)
0929 03:06:16.489662 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:16.489679 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:16.489800 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:16.490045 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:16.490047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:16.490300 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:16.490425 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:16.490431 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:16.491361 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:16.491365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:16.491989 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:16.491999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:16.492526 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:16.492529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:16.492599 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:16.493236 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:16.493250 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:16.497858 (+ 4608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":112,"threads_started":1}
W20260929 03:06:16.498279 17240 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:16.489342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:47081 (local address 127.7.209.126:38051)
0929 03:06:16.489741 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:16.489746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:16.489760 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:06:16.489885 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:16.489887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:16.489937 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:16.490020 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:16.490568 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:16.491214 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:16.492134 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:16.492411 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:16.492504 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:16.498115 (+ 5611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:47081: BlockingRecv error: recv got EOF from 127.7.209.65:47081 (error 108)
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":145,"threads_started":1}
Sep 29 03:06:16 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:16 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:16.715566 17241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:16.709440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:48929)
0929 03:06:16.709811 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:16.709828 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:16.709959 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:16.710212 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:16.710215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:16.710450 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:16.710567 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:16.710573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:16.711389 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:16.711393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:16.712043 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:16.712055 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:16.712515 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:16.712518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:16.712593 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:16.713190 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:16.713203 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:16.715390 (+ 2187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":166,"threads_started":1}
W20260929 03:06:16.715711 17240 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:16.709552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:48929 (local address 127.7.209.126:38051)
0929 03:06:16.709754 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:16.709759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:16.709904 (+ 145us) server_negotiation.cc:407] Connection header received
0929 03:06:16.710056 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:16.710058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:16.710115 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:16.710177 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:16.710679 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:16.711266 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:16.712161 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:16.712402 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:16.712487 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:16.715551 (+ 3064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:48929: BlockingRecv error: recv got EOF from 127.7.209.66:48929 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Sep 29 03:06:16 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:16 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:16.938050 17242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:16.928208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:33117)
0929 03:06:16.928534 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:16.928549 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:16.928671 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:16.928909 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:16.928911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:16.929138 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:16.929254 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:16.929261 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:16.930076 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:16.930081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:16.930687 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:16.930695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:16.931125 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:16.931128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:16.931194 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:16.935705 (+ 4511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:16.935720 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:16.937892 (+ 2172us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":115,"threads_started":1}
W20260929 03:06:16.938216 17240 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:16.928394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:33117 (local address 127.7.209.126:38051)
0929 03:06:16.928537 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:16.928541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:16.928608 (+ 67us) server_negotiation.cc:407] Connection header received
0929 03:06:16.928759 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:16.928761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:16.928815 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:16.928879 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:16.929368 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:16.929984 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:16.930794 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:16.931024 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:16.931104 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:16.938080 (+ 6976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:33117: BlockingRecv error: recv got EOF from 127.7.209.67:33117 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 29 03:06:17 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:17 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:17.505173 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:17.499102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:49927)
0929 03:06:17.499591 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:17.499608 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:17.499758 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:17.500041 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:17.500045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:17.500285 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:17.500400 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:17.500407 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:17.501224 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:17.501228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:17.501869 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:17.501878 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:17.502364 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:17.502371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:17.502474 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:17.503040 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:17.503052 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:17.505000 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":126,"threads_started":1}
W20260929 03:06:17.505313 17244 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:17.499212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:49927 (local address 127.7.209.126:38051)
0929 03:06:17.499492 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:17.499496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:17.499695 (+ 199us) server_negotiation.cc:407] Connection header received
0929 03:06:17.499859 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:17.499862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:17.499922 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:17.500003 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:17.500521 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:17.501113 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:17.501983 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:17.502242 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:17.502337 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:17.505169 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:49927: BlockingRecv error: recv got EOF from 127.7.209.65:49927 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":72,"threads_started":1}
Sep 29 03:06:17 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:17 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:17.722411 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:17.716444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:41965)
0929 03:06:17.716786 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:17.716803 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:17.716939 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:17.717201 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:17.717204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:17.717457 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:17.717569 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:17.717576 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:17.718399 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:17.718401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:17.719029 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:17.719037 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:17.719423 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:17.719425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:17.719484 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:17.720029 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:17.720039 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:17.722265 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
W20260929 03:06:17.722525 17244 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:17.716629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:41965 (local address 127.7.209.126:38051)
0929 03:06:17.716789 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:17.716794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:17.716878 (+ 84us) server_negotiation.cc:407] Connection header received
0929 03:06:17.717031 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:17.717034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:17.717092 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:17.717161 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:17.717727 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:17.718311 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:17.719118 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:17.719341 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:17.719414 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:17.722404 (+ 2990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:41965: BlockingRecv error: recv got EOF from 127.7.209.66:41965 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 29 03:06:17 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:17 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:17.944957 17246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:17.938961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:50899)
0929 03:06:17.939290 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:17.939306 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:17.939427 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:17.939744 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:17.939747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:17.939994 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:17.940134 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:17.940143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:17.941152 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:17.941156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:17.941857 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:17.941869 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:17.942261 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:17.942263 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:17.942331 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:17.942854 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:17.942866 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:17.944805 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":119,"threads_started":1}
W20260929 03:06:17.945070 17244 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:17.939076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:50899 (local address 127.7.209.126:38051)
0929 03:06:17.939255 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:17.939260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:17.939379 (+ 119us) server_negotiation.cc:407] Connection header received
0929 03:06:17.939518 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:17.939521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:17.939589 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:17.939696 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:17.940262 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:17.941052 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:17.941955 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:17.942179 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:17.942251 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:17.944948 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:50899: BlockingRecv error: recv got EOF from 127.7.209.67:50899 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 29 03:06:18 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:18 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:18.512022 17247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:18.506230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:45347)
0929 03:06:18.506543 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:18.506563 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:18.506721 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:18.506982 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:18.506984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:18.507223 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:18.507351 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:18.507357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:18.508142 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:18.508151 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:18.508770 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:18.508779 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:18.509166 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:18.509168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:18.509231 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:18.509857 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:18.509871 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:18.511871 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":97,"threads_started":1}
W20260929 03:06:18.512172 17248 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:18.506267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:45347 (local address 127.7.209.126:38051)
0929 03:06:18.506615 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:18.506621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:18.506637 (+ 16us) server_negotiation.cc:407] Connection header received
0929 03:06:18.506816 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:18.506818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:18.506871 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:18.506954 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:18.507441 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:18.508061 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:18.508857 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:18.509085 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:18.509156 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:18.512014 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:45347: BlockingRecv error: recv got EOF from 127.7.209.65:45347 (error 108)
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":75,"threads_started":1}
Sep 29 03:06:18 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:18 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:18.728823 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:18.723236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:45335)
0929 03:06:18.723532 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:18.723546 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:18.723663 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:18.723913 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:18.723915 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:18.724137 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:18.724252 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:18.724257 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:18.725044 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:18.725046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:18.725651 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:18.725659 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:18.726101 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:18.726105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:18.726180 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:18.726763 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:18.726775 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:18.728684 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":91,"threads_started":1}
W20260929 03:06:18.728978 17248 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:18.723305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:45335 (local address 127.7.209.126:38051)
0929 03:06:18.723482 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:18.723486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:18.723634 (+ 148us) server_negotiation.cc:407] Connection header received
0929 03:06:18.723749 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:18.723751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:18.723801 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:18.723880 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:18.724352 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:18.724952 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:18.725743 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:18.725988 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:18.726063 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:18.728848 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:45335: BlockingRecv error: recv got EOF from 127.7.209.66:45335 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 29 03:06:18 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:18 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.67@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:18.951769 17250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:18.945923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.67:47115)
0929 03:06:18.946253 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:18.946272 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:18.946423 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:18.946735 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:18.946739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:18.947019 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:18.947173 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:18.947184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:18.948115 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:18.948118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:18.948749 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:18.948758 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:18.949145 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:18.949148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:18.949204 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:18.949759 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:18.949770 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:18.951616 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":92,"threads_started":1}
W20260929 03:06:18.951898 17248 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:18.946040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:47115 (local address 127.7.209.126:38051)
0929 03:06:18.946239 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:18.946243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:18.946356 (+ 113us) server_negotiation.cc:407] Connection header received
0929 03:06:18.946529 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:18.946533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:18.946601 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:18.946677 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:18.947299 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:18.948010 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:18.948843 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:18.949066 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:18.949137 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:18.951773 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.67:47115: BlockingRecv error: recv got EOF from 127.7.209.67:47115 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 29 03:06:19 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:19 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.65@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:19.519482 17251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:19.513168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.65:52303)
0929 03:06:19.513491 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:19.513509 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:19.513652 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:19.513983 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:19.513986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:19.514239 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:19.514375 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:19.514385 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:19.515459 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:19.515463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:19.516086 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:19.516096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:19.516521 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:19.516523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:19.516589 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:19.517194 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:19.517207 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:19.519297 (+ 2090us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":93,"threads_started":1}
W20260929 03:06:19.519583 17252 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:19.513303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:52303 (local address 127.7.209.126:38051)
0929 03:06:19.513661 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:19.513666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:19.513685 (+ 19us) server_negotiation.cc:407] Connection header received
0929 03:06:19.513765 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:19.513767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:19.513830 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:19.513934 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:19.514512 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:19.515336 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:19.516197 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:19.516434 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:19.516509 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:19.519450 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.65:52303: BlockingRecv error: recv got EOF from 127.7.209.65:52303 (error 108)
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":171,"threads_started":1}
Sep 29 03:06:19 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
Sep 29 03:06:19 dist-test-slave-bdps krb5kdc[16193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.209.66@KRBTEST.COM for oryx/127.7.209.126@KRBTEST.COM, Server not found in Kerberos database
I20260929 03:06:19.735723 17253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:19.729852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38051 (local address 127.7.209.66:48671)
0929 03:06:19.730157 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:19.730173 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:19.730282 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:19.730507 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:19.730509 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:19.730725 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:19.730840 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:19.730845 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:19.731597 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:19.731599 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:19.732249 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:19.732258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:19.732661 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:19.732664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:19.732730 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:19.733263 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:19.733277 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:19.735510 (+ 2233us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.209.126:38051: Server oryx/127.7.209.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":87,"threads_started":1}
W20260929 03:06:19.735791 17252 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:19.729965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:48671 (local address 127.7.209.126:38051)
0929 03:06:19.730154 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:19.730159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:19.730230 (+ 71us) server_negotiation.cc:407] Connection header received
0929 03:06:19.730365 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:19.730368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:19.730415 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:19.730475 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:19.730936 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:19.731509 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:19.732343 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:19.732566 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:19.732638 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:19.735678 (+ 3040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.209.66:48671: BlockingRecv error: recv got EOF from 127.7.209.66:48671 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 03:06:19.825867 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 16776
I20260929 03:06:19.833412 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 16928
I20260929 03:06:19.839046 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 17060
I20260929 03:06:19.844262 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 16706
2026-09-29T03:06:19Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18155 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T03:06:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:06:19Z Disabled control of system clock
I20260929 03:06:19.878235 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:39847
--webserver_interface=127.7.209.126
--webserver_port=0
--builtin_ntp_servers=127.7.209.84:36187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:39847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 03:06:20.010346 17257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:20.010725 17257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:20.010766 17257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:20.015447 17257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:06:20.015561 17257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:20.015575 17257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:20.015592 17257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:06:20.015607 17257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:06:20.020936 17257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:36187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:39847
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:39847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:20.022475 17257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:20.024271 17257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:20.034145 17262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:20.034158 17263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:20.034214 17265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:20.034268 17257 server_base.cc:1034] running on GCE node
I20260929 03:06:20.034924 17257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:20.036142 17257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:20.037314 17257 hybrid_clock.cc:648] HybridClock initialized: now 1790651180037294 us; error 31 us; skew 500 ppm
I20260929 03:06:20.039330 17257 webserver.cc:460] Webserver started at http://127.7.209.126:37679/ using document root <none> and password file <none>
I20260929 03:06:20.039958 17257 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:20.040030 17257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:20.040266 17257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:20.042577 17257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "9c08cb907cb5426d8e74be33b87a7e33"
format_stamp: "Formatted at 2026-09-29 03:06:20 on dist-test-slave-bdps"
server_key: "3a11eea3038f32c33adab65a853ba542"
server_key_iv: "f0369614102a128cd8acf46575fa4f3a"
server_key_version: "encryptionkey@0"
I20260929 03:06:20.043148 17257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "9c08cb907cb5426d8e74be33b87a7e33"
format_stamp: "Formatted at 2026-09-29 03:06:20 on dist-test-slave-bdps"
server_key: "3a11eea3038f32c33adab65a853ba542"
server_key_iv: "f0369614102a128cd8acf46575fa4f3a"
server_key_version: "encryptionkey@0"
I20260929 03:06:20.047051 17257 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:06:20.049528 17271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:20.050864 17257 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 03:06:20.050980 17257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "9c08cb907cb5426d8e74be33b87a7e33"
format_stamp: "Formatted at 2026-09-29 03:06:20 on dist-test-slave-bdps"
server_key: "3a11eea3038f32c33adab65a853ba542"
server_key_iv: "f0369614102a128cd8acf46575fa4f3a"
server_key_version: "encryptionkey@0"
I20260929 03:06:20.051134 17257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:20.086387 17257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:20.087293 17257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:20.087486 17257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:20.096230 17257 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:39847
I20260929 03:06:20.096235 17323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:39847 every 8 connection(s)
I20260929 03:06:20.097503 17257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:06:20.101361 17324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:06:20.104630 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 17257
I20260929 03:06:20.104753 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:06:20.105069 8005 external_mini_cluster.cc:1428] Setting key 103bc48929a518e910f09c70af118f68
I20260929 03:06:20.108342 17324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33: Bootstrap starting.
I20260929 03:06:20.110991 17324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33: Neither blocks nor log segments found. Creating new log.
I20260929 03:06:20.111806 17324 log.cc:824] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33: Log is configured to *not* fsync() on all Append() calls
I20260929 03:06:20.113847 17324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33: No bootstrap required, opened a new log
I20260929 03:06:20.115438 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:20.106914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52182 (local address 127.7.209.126:39847)
0929 03:06:20.107690 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:20.107710 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:20.107753 (+ 43us) server_negotiation.cc:407] Connection header received
0929 03:06:20.108442 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:20.108463 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:20.108958 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:20.109331 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:20.109870 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.110919 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.111899 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.112437 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.112594 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:20.113493 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:20.113498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:20.113513 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:06:20.114017 (+ 504us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:20.114087 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:06:20.114110 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:20.114209 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:20.114348 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 03:06:20.114629 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":160,"threads_started":1}
I20260929 03:06:20.118057 17324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c08cb907cb5426d8e74be33b87a7e33" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39847 } }
I20260929 03:06:20.118305 17324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:06:20.118362 17324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c08cb907cb5426d8e74be33b87a7e33, State: Initialized, Role: FOLLOWER
I20260929 03:06:20.118796 17324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c08cb907cb5426d8e74be33b87a7e33" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39847 } }
I20260929 03:06:20.118929 17324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:06:20.118991 17324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:06:20.119091 17324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:20.119951 17324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c08cb907cb5426d8e74be33b87a7e33" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39847 } }
I20260929 03:06:20.120306 17324 leader_election.cc:302] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c08cb907cb5426d8e74be33b87a7e33; no voters:
I20260929 03:06:20.120589 17324 leader_election.cc:288] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:06:20.120673 17329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:06:20.120936 17329 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [term 1 LEADER]: Becoming Leader. State: Replica: 9c08cb907cb5426d8e74be33b87a7e33, State: Running, Role: LEADER
I20260929 03:06:20.121326 17329 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c08cb907cb5426d8e74be33b87a7e33" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39847 } }
I20260929 03:06:20.121941 17324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:06:20.123154 17330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c08cb907cb5426d8e74be33b87a7e33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c08cb907cb5426d8e74be33b87a7e33" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39847 } } }
I20260929 03:06:20.123292 17330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [sys.catalog]: This master's current role is: LEADER
I20260929 03:06:20.123714 17338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:06:20.123641 17331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c08cb907cb5426d8e74be33b87a7e33. Latest consensus state: current_term: 1 leader_uuid: "9c08cb907cb5426d8e74be33b87a7e33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c08cb907cb5426d8e74be33b87a7e33" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39847 } } }
I20260929 03:06:20.124279 17331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33 [sys.catalog]: This master's current role is: LEADER
I20260929 03:06:20.126794 17338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:06:20.133101 17338 catalog_manager.cc:1347] Generated new cluster ID: ef33242a597349b79972654f5cffa5ef
I20260929 03:06:20.133178 17338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:06:20.147960 17338 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:06:20.149358 17338 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:06:20.156373 17338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c08cb907cb5426d8e74be33b87a7e33: Generated new TSK 0
I20260929 03:06:20.157155 17338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:06:20.161904 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39847
--builtin_ntp_servers=127.7.209.84:36187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 03:06:20.289301 17348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:20.289657 17348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:20.289724 17348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:20.293859 17348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:20.293926 17348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:20.293983 17348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:06:20.299791 17348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:36187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:20.301482 17348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:20.303349 17348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:20.314337 17356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:20.314429 17348 server_base.cc:1034] running on GCE node
W20260929 03:06:20.314360 17354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:20.314394 17353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:20.315013 17348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:20.315758 17348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:20.316908 17348 hybrid_clock.cc:648] HybridClock initialized: now 1790651180316882 us; error 43 us; skew 500 ppm
I20260929 03:06:20.318917 17348 webserver.cc:460] Webserver started at http://127.7.209.65:35357/ using document root <none> and password file <none>
I20260929 03:06:20.319550 17348 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:20.319612 17348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:20.319830 17348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:20.322220 17348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "73081e14f04a4e2f8b55b5553a963e53"
format_stamp: "Formatted at 2026-09-29 03:06:20 on dist-test-slave-bdps"
server_key: "e07d4c3574e8d3c583ea5e14ecfd9d4b"
server_key_iv: "462dc282393e0f181d9ac7e06c40a08d"
server_key_version: "encryptionkey@0"
I20260929 03:06:20.322878 17348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "73081e14f04a4e2f8b55b5553a963e53"
format_stamp: "Formatted at 2026-09-29 03:06:20 on dist-test-slave-bdps"
server_key: "e07d4c3574e8d3c583ea5e14ecfd9d4b"
server_key_iv: "462dc282393e0f181d9ac7e06c40a08d"
server_key_version: "encryptionkey@0"
I20260929 03:06:20.327045 17348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:06:20.329730 17362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:20.331110 17348 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260929 03:06:20.331215 17348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "73081e14f04a4e2f8b55b5553a963e53"
format_stamp: "Formatted at 2026-09-29 03:06:20 on dist-test-slave-bdps"
server_key: "e07d4c3574e8d3c583ea5e14ecfd9d4b"
server_key_iv: "462dc282393e0f181d9ac7e06c40a08d"
server_key_version: "encryptionkey@0"
I20260929 03:06:20.331311 17348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:20.347633 17348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:20.348493 17348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:20.348682 17348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:20.349300 17348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:20.350389 17348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:20.350438 17348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:20.350485 17348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:20.350513 17348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:20.360479 17348 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:40323
I20260929 03:06:20.360551 17475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:40323 every 8 connection(s)
I20260929 03:06:20.361513 17348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:06:20.370751 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 17348
I20260929 03:06:20.370886 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:06:20.371240 8005 external_mini_cluster.cc:1428] Setting key ca57661f5ec2f9efa9c0743ec6d7b761
I20260929 03:06:20.371407 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:20.363578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:46431 (local address 127.7.209.126:39847)
0929 03:06:20.363743 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:20.363748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:20.364978 (+ 1230us) server_negotiation.cc:407] Connection header received
0929 03:06:20.366485 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:20.366488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:20.366548 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:20.366631 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:20.367570 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.368197 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.369324 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.369638 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.369749 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:20.370874 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:20.370878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:20.370882 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:06:20.370895 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:20.370919 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:06:20.370931 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:20.370998 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:20.371162 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 03:06:20.371237 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 03:06:20.372437 17478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:20.364076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39847 (local address 127.7.209.65:46431)
0929 03:06:20.364817 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:20.364862 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:20.366287 (+ 1425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:20.366807 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:20.366814 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:20.366909 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:20.367454 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:20.367468 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.368295 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.368297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:20.369232 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:20.369240 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.369785 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.369787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:20.369942 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:20.370551 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:06:20.370553 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:20.370614 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:06:20.370710 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:06:20.371027 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:06:20.371036 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:06:20.371040 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 03:06:20.371156 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 03:06:20.371472 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":130,"threads_started":1}
I20260929 03:06:20.373453 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39847
--builtin_ntp_servers=127.7.209.84:36187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 03:06:20.373694 17476 heartbeater.cc:347] Connected to a master server at 127.7.209.126:39847
I20260929 03:06:20.373944 17476 heartbeater.cc:464] Registering TS with master...
I20260929 03:06:20.374631 17476 heartbeater.cc:511] Master 127.7.209.126:39847 requested a full tablet report, sending...
I20260929 03:06:20.376788 17288 ts_manager.cc:194] Registered new tserver with Master: 73081e14f04a4e2f8b55b5553a963e53 (127.7.209.65:40323)
I20260929 03:06:20.378433 17288 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.209.65:46431
W20260929 03:06:20.502815 17479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:20.503142 17479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:20.503197 17479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:20.507340 17479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:20.507393 17479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:20.507467 17479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:06:20.512322 17479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:36187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:20.513538 17479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:20.515187 17479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:20.525796 17487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:20.525852 17485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:20.525882 17484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:20.526082 17479 server_base.cc:1034] running on GCE node
I20260929 03:06:20.526504 17479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:20.527184 17479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:20.528338 17479 hybrid_clock.cc:648] HybridClock initialized: now 1790651180528322 us; error 29 us; skew 500 ppm
I20260929 03:06:20.530275 17479 webserver.cc:460] Webserver started at http://127.7.209.66:33849/ using document root <none> and password file <none>
I20260929 03:06:20.530889 17479 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:20.530953 17479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:20.531164 17479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:20.533406 17479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "f9d2abd3f0c64e889e557a4eee253d43"
format_stamp: "Formatted at 2026-09-29 03:06:20 on dist-test-slave-bdps"
server_key: "7855701b7ee9ebf86ebd43276ff65f6f"
server_key_iv: "1989c90170ff500379f452e95d1a0d4b"
server_key_version: "encryptionkey@0"
I20260929 03:06:20.534051 17479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "f9d2abd3f0c64e889e557a4eee253d43"
format_stamp: "Formatted at 2026-09-29 03:06:20 on dist-test-slave-bdps"
server_key: "7855701b7ee9ebf86ebd43276ff65f6f"
server_key_iv: "1989c90170ff500379f452e95d1a0d4b"
server_key_version: "encryptionkey@0"
I20260929 03:06:20.538482 17479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:06:20.541332 17493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:20.542754 17479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:06:20.542865 17479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "f9d2abd3f0c64e889e557a4eee253d43"
format_stamp: "Formatted at 2026-09-29 03:06:20 on dist-test-slave-bdps"
server_key: "7855701b7ee9ebf86ebd43276ff65f6f"
server_key_iv: "1989c90170ff500379f452e95d1a0d4b"
server_key_version: "encryptionkey@0"
I20260929 03:06:20.542970 17479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:20.556890 17479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:20.557552 17479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:20.557747 17479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:20.558332 17479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:20.559315 17479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:20.559363 17479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:20.559414 17479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:20.559440 17479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:20.568876 17479 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:39995
I20260929 03:06:20.568910 17606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:39995 every 8 connection(s)
I20260929 03:06:20.569903 17479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:06:20.572183 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 17479
I20260929 03:06:20.572306 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:06:20.572649 8005 external_mini_cluster.cc:1428] Setting key 527f5a3154c3c1d24497690d45dc7545
I20260929 03:06:20.575043 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39847
--builtin_ntp_servers=127.7.209.84:36187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 03:06:20.581326 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:20.571908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:52781 (local address 127.7.209.126:39847)
0929 03:06:20.572081 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:20.572087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:20.573417 (+ 1330us) server_negotiation.cc:407] Connection header received
0929 03:06:20.575139 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:20.575142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:20.575208 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:20.575304 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:20.576442 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.577248 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.578981 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.579323 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.579433 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:20.580732 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:20.580737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:20.580740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:06:20.580759 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:20.580792 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:06:20.580805 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:20.580889 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:20.581027 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 03:06:20.581116 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:06:20.582456 17609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:20.572418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39847 (local address 127.7.209.66:52781)
0929 03:06:20.573244 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:20.573303 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:20.574883 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:20.575529 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:20.575538 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:20.575646 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:20.576280 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:20.576298 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.577386 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.577391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:20.578845 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:20.578857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.579482 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.579485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:20.579657 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:20.580423 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:06:20.580425 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:20.580477 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:06:20.580561 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:06:20.580886 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:06:20.580893 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:06:20.580897 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 03:06:20.581009 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 03:06:20.581305 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":129,"threads_started":1}
I20260929 03:06:20.583801 17607 heartbeater.cc:347] Connected to a master server at 127.7.209.126:39847
I20260929 03:06:20.584093 17607 heartbeater.cc:464] Registering TS with master...
I20260929 03:06:20.584802 17607 heartbeater.cc:511] Master 127.7.209.126:39847 requested a full tablet report, sending...
I20260929 03:06:20.586063 17288 ts_manager.cc:194] Registered new tserver with Master: f9d2abd3f0c64e889e557a4eee253d43 (127.7.209.66:39995)
I20260929 03:06:20.586746 17288 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.209.66:52781
W20260929 03:06:20.706832 17610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:20.707147 17610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:20.707185 17610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:20.711315 17610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:20.711366 17610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:20.711426 17610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:06:20.716408 17610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:36187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:20.717801 17610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:20.719521 17610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:20.730522 17615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:20.730491 17616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:20.730573 17610 server_base.cc:1034] running on GCE node
W20260929 03:06:20.730499 17618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:20.731096 17610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:20.731813 17610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:20.732980 17610 hybrid_clock.cc:648] HybridClock initialized: now 1790651180732964 us; error 31 us; skew 500 ppm
I20260929 03:06:20.735167 17610 webserver.cc:460] Webserver started at http://127.7.209.67:39183/ using document root <none> and password file <none>
I20260929 03:06:20.735886 17610 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:20.735960 17610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:20.736210 17610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:20.738763 17610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "a7213e3c9df441afa5d097df347b6ef7"
format_stamp: "Formatted at 2026-09-29 03:06:20 on dist-test-slave-bdps"
server_key: "b2233c7d10b710b407861b169e585cf8"
server_key_iv: "23bf3db2df96107c8a6a81b3927b2cbc"
server_key_version: "encryptionkey@0"
I20260929 03:06:20.739394 17610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "a7213e3c9df441afa5d097df347b6ef7"
format_stamp: "Formatted at 2026-09-29 03:06:20 on dist-test-slave-bdps"
server_key: "b2233c7d10b710b407861b169e585cf8"
server_key_iv: "23bf3db2df96107c8a6a81b3927b2cbc"
server_key_version: "encryptionkey@0"
I20260929 03:06:20.743808 17610 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 03:06:20.746726 17624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:20.748170 17610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:06:20.748281 17610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "a7213e3c9df441afa5d097df347b6ef7"
format_stamp: "Formatted at 2026-09-29 03:06:20 on dist-test-slave-bdps"
server_key: "b2233c7d10b710b407861b169e585cf8"
server_key_iv: "23bf3db2df96107c8a6a81b3927b2cbc"
server_key_version: "encryptionkey@0"
I20260929 03:06:20.748379 17610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:20.767278 17610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:20.767983 17610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:20.768147 17610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:20.768733 17610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:20.769744 17610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:20.769793 17610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:20.769850 17610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:20.769876 17610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:20.779881 17610 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:39347
I20260929 03:06:20.779903 17737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:39347 every 8 connection(s)
I20260929 03:06:20.781056 17610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:06:20.784247 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 17610
I20260929 03:06:20.784377 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:06:20.784756 8005 external_mini_cluster.cc:1428] Setting key 980916573a9d3a9e2dac313cb47276d2
I20260929 03:06:20.791563 17327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:20.783292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:34129 (local address 127.7.209.126:39847)
0929 03:06:20.783487 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:20.783492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:20.784926 (+ 1434us) server_negotiation.cc:407] Connection header received
0929 03:06:20.786642 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:20.786645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:20.786714 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:20.786792 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:20.787823 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.788395 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.789582 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.789865 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.789958 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:20.791083 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:20.791086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:20.791089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:06:20.791101 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:20.791125 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:06:20.791133 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:20.791191 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:20.791335 (+ 144us) server_negotiation.cc:299] Negotiation successful
0929 03:06:20.791404 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 03:06:20.792495 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:20.783846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39847 (local address 127.7.209.67:34129)
0929 03:06:20.784746 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:20.784793 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:20.786436 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:20.786994 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:20.787002 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:20.787100 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:20.787689 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:20.787704 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.788514 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.788517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:20.789467 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:20.789475 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:20.790012 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:20.790016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:20.790194 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:20.790821 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:06:20.790823 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:20.790869 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:06:20.790938 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:06:20.791206 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:06:20.791215 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:06:20.791220 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 03:06:20.791330 (+ 110us) client_negotiation.cc:240] Negotiation successful
0929 03:06:20.791638 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":164,"threads_started":1}
I20260929 03:06:20.793557 17738 heartbeater.cc:347] Connected to a master server at 127.7.209.126:39847
I20260929 03:06:20.793803 17738 heartbeater.cc:464] Registering TS with master...
I20260929 03:06:20.794337 17738 heartbeater.cc:511] Master 127.7.209.126:39847 requested a full tablet report, sending...
I20260929 03:06:20.795359 17288 ts_manager.cc:194] Registered new tserver with Master: a7213e3c9df441afa5d097df347b6ef7 (127.7.209.67:39347)
I20260929 03:06:20.795964 17288 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.209.67:34129
I20260929 03:06:20.798626 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 03:06:20.801527 17327 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:20.800739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52184 (local address 127.7.209.126:39847)
0929 03:06:20.800852 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:20.800855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:20.801060 (+ 205us) server_negotiation.cc:407] Connection header received
0929 03:06:20.801154 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:20.801156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:20.801193 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:20.801252 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:20.801431 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52184: BlockingRecv error: recv got EOF from 127.0.0.1:52184 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:06:20.801980 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 17348
I20260929 03:06:20.807813 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 17479
I20260929 03:06:20.812809 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 17610
I20260929 03:06:20.817497 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 17257
2026-09-29T03:06:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (970 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T03:06:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:06:20Z Disabled control of system clock
I20260929 03:06:20.846832 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:40741
--webserver_interface=127.7.209.126
--webserver_port=0
--builtin_ntp_servers=127.7.209.84:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:40741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 03:06:20.973363 17749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:20.973731 17749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:20.973771 17749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:20.977878 17749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:06:20.977954 17749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:20.977969 17749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:20.977988 17749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:06:20.978003 17749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:06:20.983402 17749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:40741
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:40741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:20.984856 17749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:20.986673 17749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:20.996309 17755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:20.996344 17749 server_base.cc:1034] running on GCE node
W20260929 03:06:20.996285 17754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:20.996294 17757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:20.997149 17749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:20.998400 17749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:20.999562 17749 hybrid_clock.cc:648] HybridClock initialized: now 1790651180999546 us; error 31 us; skew 500 ppm
I20260929 03:06:21.001600 17749 webserver.cc:460] Webserver started at http://127.7.209.126:36283/ using document root <none> and password file <none>
I20260929 03:06:21.002323 17749 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:21.002390 17749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:21.002629 17749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:21.005049 17749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "219f7413c6524ad7aec04fcf68f394be"
format_stamp: "Formatted at 2026-09-29 03:06:21 on dist-test-slave-bdps"
server_key: "45136ae347bba65f876a3d708fb3ceb7"
server_key_iv: "6e07d885f432cc1a14e3ca8d7625dcf4"
server_key_version: "encryptionkey@0"
I20260929 03:06:21.005664 17749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "219f7413c6524ad7aec04fcf68f394be"
format_stamp: "Formatted at 2026-09-29 03:06:21 on dist-test-slave-bdps"
server_key: "45136ae347bba65f876a3d708fb3ceb7"
server_key_iv: "6e07d885f432cc1a14e3ca8d7625dcf4"
server_key_version: "encryptionkey@0"
I20260929 03:06:21.009766 17749 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:06:21.012355 17763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:21.013881 17749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:06:21.014002 17749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "219f7413c6524ad7aec04fcf68f394be"
format_stamp: "Formatted at 2026-09-29 03:06:21 on dist-test-slave-bdps"
server_key: "45136ae347bba65f876a3d708fb3ceb7"
server_key_iv: "6e07d885f432cc1a14e3ca8d7625dcf4"
server_key_version: "encryptionkey@0"
I20260929 03:06:21.014118 17749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:21.019454 17749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:21.020210 17749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:21.020388 17749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:21.028412 17749 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:40741
I20260929 03:06:21.028427 17815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:40741 every 8 connection(s)
I20260929 03:06:21.029529 17749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:06:21.033366 17816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:06:21.033380 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 17749
I20260929 03:06:21.033489 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:06:21.033874 8005 external_mini_cluster.cc:1428] Setting key 6f3940c96d918c75ad40175aa599e49d
I20260929 03:06:21.040232 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:21.035557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57066 (local address 127.7.209.126:40741)
0929 03:06:21.036256 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:21.036274 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:21.036315 (+ 41us) server_negotiation.cc:407] Connection header received
0929 03:06:21.037076 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:21.037104 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:21.037530 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:21.037928 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:21.038792 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:21.038796 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:21.038810 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:06:21.038819 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:21.038878 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:06:21.038895 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:21.039015 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:21.039122 (+ 107us) server_negotiation.cc:299] Negotiation successful
0929 03:06:21.039409 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":117,"threads_started":1}
I20260929 03:06:21.041369 17816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be: Bootstrap starting.
I20260929 03:06:21.043869 17816 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be: Neither blocks nor log segments found. Creating new log.
I20260929 03:06:21.044670 17816 log.cc:824] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be: Log is configured to *not* fsync() on all Append() calls
I20260929 03:06:21.046653 17816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be: No bootstrap required, opened a new log
I20260929 03:06:21.050489 17816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219f7413c6524ad7aec04fcf68f394be" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40741 } }
I20260929 03:06:21.050729 17816 raft_consensus.cc:374] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:06:21.050788 17816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 219f7413c6524ad7aec04fcf68f394be, State: Initialized, Role: FOLLOWER
I20260929 03:06:21.051260 17816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219f7413c6524ad7aec04fcf68f394be" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40741 } }
I20260929 03:06:21.051400 17816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:06:21.051465 17816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:06:21.051568 17816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:21.052488 17816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219f7413c6524ad7aec04fcf68f394be" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40741 } }
I20260929 03:06:21.052847 17816 leader_election.cc:302] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 219f7413c6524ad7aec04fcf68f394be; no voters:
I20260929 03:06:21.053156 17816 leader_election.cc:288] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:06:21.053259 17821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:06:21.053484 17821 raft_consensus.cc:686] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [term 1 LEADER]: Becoming Leader. State: Replica: 219f7413c6524ad7aec04fcf68f394be, State: Running, Role: LEADER
I20260929 03:06:21.053846 17821 consensus_queue.cc:237] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219f7413c6524ad7aec04fcf68f394be" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40741 } }
I20260929 03:06:21.054543 17816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:06:21.055770 17822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "219f7413c6524ad7aec04fcf68f394be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219f7413c6524ad7aec04fcf68f394be" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40741 } } }
I20260929 03:06:21.055768 17823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 219f7413c6524ad7aec04fcf68f394be. Latest consensus state: current_term: 1 leader_uuid: "219f7413c6524ad7aec04fcf68f394be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "219f7413c6524ad7aec04fcf68f394be" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40741 } } }
I20260929 03:06:21.055920 17822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [sys.catalog]: This master's current role is: LEADER
I20260929 03:06:21.055920 17823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be [sys.catalog]: This master's current role is: LEADER
I20260929 03:06:21.056391 17830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:06:21.060156 17830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:06:21.066761 17830 catalog_manager.cc:1347] Generated new cluster ID: 81c7896e98bc428695cb93be370cf5f8
I20260929 03:06:21.066825 17830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:06:21.115782 17830 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:06:21.116281 17830 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:06:21.134563 17830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 219f7413c6524ad7aec04fcf68f394be: Generated new TSK 0
I20260929 03:06:21.135514 17830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:06:21.150226 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:40741
--builtin_ntp_servers=127.7.209.84:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 03:06:21.279577 17840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:21.279897 17840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:21.279938 17840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:21.284058 17840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:21.284121 17840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:21.284198 17840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:06:21.289014 17840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:40741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:21.290310 17840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:21.291983 17840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:21.303908 17845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:21.304155 17848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:21.303908 17846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:21.304667 17840 server_base.cc:1034] running on GCE node
I20260929 03:06:21.305130 17840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:21.305923 17840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:21.307078 17840 hybrid_clock.cc:648] HybridClock initialized: now 1790651181307049 us; error 62 us; skew 500 ppm
I20260929 03:06:21.309422 17840 webserver.cc:460] Webserver started at http://127.7.209.65:45919/ using document root <none> and password file <none>
I20260929 03:06:21.310209 17840 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:21.310278 17840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:21.310521 17840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:21.313051 17840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "6c61e108e6704701a93c0ab2f6fff96d"
format_stamp: "Formatted at 2026-09-29 03:06:21 on dist-test-slave-bdps"
server_key: "c1e1c7de1d481f94852a620a8093e4c9"
server_key_iv: "f1e6b8b1565296382bac8ac6d90c10b3"
server_key_version: "encryptionkey@0"
I20260929 03:06:21.313774 17840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "6c61e108e6704701a93c0ab2f6fff96d"
format_stamp: "Formatted at 2026-09-29 03:06:21 on dist-test-slave-bdps"
server_key: "c1e1c7de1d481f94852a620a8093e4c9"
server_key_iv: "f1e6b8b1565296382bac8ac6d90c10b3"
server_key_version: "encryptionkey@0"
I20260929 03:06:21.318218 17840 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:06:21.321141 17854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:21.322561 17840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:06:21.322676 17840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "6c61e108e6704701a93c0ab2f6fff96d"
format_stamp: "Formatted at 2026-09-29 03:06:21 on dist-test-slave-bdps"
server_key: "c1e1c7de1d481f94852a620a8093e4c9"
server_key_iv: "f1e6b8b1565296382bac8ac6d90c10b3"
server_key_version: "encryptionkey@0"
I20260929 03:06:21.322780 17840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:21.327818 17840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:21.328485 17840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:21.328646 17840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:21.329249 17840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:21.330312 17840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:21.330359 17840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:21.330411 17840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:21.330440 17840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:21.340169 17840 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:37161
I20260929 03:06:21.340194 17967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:37161 every 8 connection(s)
I20260929 03:06:21.341315 17840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:06:21.348059 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:21.343517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:45973 (local address 127.7.209.126:40741)
0929 03:06:21.343703 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:21.343708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:21.344998 (+ 1290us) server_negotiation.cc:407] Connection header received
0929 03:06:21.346440 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:21.346442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:21.346484 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:21.346563 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:21.347700 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:21.347702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:21.347704 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:06:21.347712 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:21.347729 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:06:21.347736 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:21.347804 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:21.347891 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 03:06:21.347930 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 03:06:21.348279 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 17840
I20260929 03:06:21.348412 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:06:21.348805 8005 external_mini_cluster.cc:1428] Setting key ebcbedf4376235beaf004820aab9cee3
I20260929 03:06:21.349277 17970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:21.344047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:40741 (local address 127.7.209.65:45973)
0929 03:06:21.344825 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:21.344872 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:21.346233 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:21.346723 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:21.346730 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:21.346833 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:21.347424 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:06:21.347426 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:21.347475 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:06:21.347553 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:06:21.347788 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:06:21.347792 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:06:21.347796 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 03:06:21.347925 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 03:06:21.348205 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":139,"threads_started":1}
I20260929 03:06:21.350553 17968 heartbeater.cc:347] Connected to a master server at 127.7.209.126:40741
I20260929 03:06:21.350788 17968 heartbeater.cc:464] Registering TS with master...
I20260929 03:06:21.351043 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:40741
--builtin_ntp_servers=127.7.209.84:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 03:06:21.351230 17968 heartbeater.cc:511] Master 127.7.209.126:40741 requested a full tablet report, sending...
I20260929 03:06:21.354552 17780 ts_manager.cc:194] Registered new tserver with Master: 6c61e108e6704701a93c0ab2f6fff96d (127.7.209.65:37161)
W20260929 03:06:21.489305 17971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:21.489662 17971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:21.489705 17971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:21.493911 17971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:21.494004 17971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:21.494078 17971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:06:21.500196 17971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:40741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:21.501775 17971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:21.503683 17971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:21.515005 17977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:21.515029 17976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:21.515029 17979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:21.515072 17971 server_base.cc:1034] running on GCE node
I20260929 03:06:21.515774 17971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:21.516515 17971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:21.517679 17971 hybrid_clock.cc:648] HybridClock initialized: now 1790651181517660 us; error 30 us; skew 500 ppm
I20260929 03:06:21.519747 17971 webserver.cc:460] Webserver started at http://127.7.209.66:38325/ using document root <none> and password file <none>
I20260929 03:06:21.520392 17971 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:21.520453 17971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:21.520671 17971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:21.523128 17971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "819950d113ef4eebb7b47fccb2de3fca"
format_stamp: "Formatted at 2026-09-29 03:06:21 on dist-test-slave-bdps"
server_key: "7940908750d46af28f8fb1939dd4b436"
server_key_iv: "c187e6127084de8010de0472e84e5f64"
server_key_version: "encryptionkey@0"
I20260929 03:06:21.523705 17971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "819950d113ef4eebb7b47fccb2de3fca"
format_stamp: "Formatted at 2026-09-29 03:06:21 on dist-test-slave-bdps"
server_key: "7940908750d46af28f8fb1939dd4b436"
server_key_iv: "c187e6127084de8010de0472e84e5f64"
server_key_version: "encryptionkey@0"
I20260929 03:06:21.528126 17971 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:06:21.530982 17985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:21.532366 17971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:06:21.532480 17971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "819950d113ef4eebb7b47fccb2de3fca"
format_stamp: "Formatted at 2026-09-29 03:06:21 on dist-test-slave-bdps"
server_key: "7940908750d46af28f8fb1939dd4b436"
server_key_iv: "c187e6127084de8010de0472e84e5f64"
server_key_version: "encryptionkey@0"
I20260929 03:06:21.532585 17971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:21.537899 17971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:21.538666 17971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:21.538856 17971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:21.539527 17971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:21.540616 17971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:21.540665 17971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:21.540720 17971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:21.540748 17971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:21.550421 17971 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:40221
I20260929 03:06:21.550452 18098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:40221 every 8 connection(s)
I20260929 03:06:21.551477 17971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:06:21.558638 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:21.553667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:42803 (local address 127.7.209.126:40741)
0929 03:06:21.553850 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:21.553855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:21.555275 (+ 1420us) server_negotiation.cc:407] Connection header received
0929 03:06:21.556796 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:21.556800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:21.556878 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:21.556967 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:21.558219 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:21.558222 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:21.558224 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:06:21.558237 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:21.558261 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:06:21.558269 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:21.558363 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:21.558451 (+ 88us) server_negotiation.cc:299] Negotiation successful
0929 03:06:21.558495 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 03:06:21.559602 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:21.554246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:40741 (local address 127.7.209.66:42803)
0929 03:06:21.555077 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:21.555123 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:21.556556 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:21.557175 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:21.557184 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:21.557290 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:21.557901 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:06:21.557902 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:21.557952 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:06:21.558030 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:06:21.558344 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:06:21.558349 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:06:21.558354 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 03:06:21.558484 (+ 130us) client_negotiation.cc:240] Negotiation successful
0929 03:06:21.558709 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":140,"threads_started":1}
I20260929 03:06:21.560142 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 17971
I20260929 03:06:21.560292 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:06:21.560683 8005 external_mini_cluster.cc:1428] Setting key 536abaad7afe40d8a5a59bb9b7fe9e1c
I20260929 03:06:21.560914 18099 heartbeater.cc:347] Connected to a master server at 127.7.209.126:40741
I20260929 03:06:21.561172 18099 heartbeater.cc:464] Registering TS with master...
I20260929 03:06:21.561712 18099 heartbeater.cc:511] Master 127.7.209.126:40741 requested a full tablet report, sending...
I20260929 03:06:21.562847 17780 ts_manager.cc:194] Registered new tserver with Master: 819950d113ef4eebb7b47fccb2de3fca (127.7.209.66:40221)
I20260929 03:06:21.563047 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:40741
--builtin_ntp_servers=127.7.209.84:40419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 03:06:21.692411 18102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:21.692700 18102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:21.692736 18102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:21.696741 18102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:21.696831 18102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:21.696887 18102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:06:21.701822 18102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:40419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:40741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:21.703184 18102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:21.704914 18102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:21.715623 18108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:21.715665 18107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:21.715665 18110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:21.715863 18102 server_base.cc:1034] running on GCE node
I20260929 03:06:21.716325 18102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:21.716997 18102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:21.718145 18102 hybrid_clock.cc:648] HybridClock initialized: now 1790651181718117 us; error 43 us; skew 500 ppm
I20260929 03:06:21.720099 18102 webserver.cc:460] Webserver started at http://127.7.209.67:35659/ using document root <none> and password file <none>
I20260929 03:06:21.720726 18102 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:21.720798 18102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:21.721020 18102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:21.723318 18102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "31a294676da940f1b17af6b92dc2e333"
format_stamp: "Formatted at 2026-09-29 03:06:21 on dist-test-slave-bdps"
server_key: "6203f297615594f1804c568d5c2bc86c"
server_key_iv: "54e54ccf57d338b6693575055e73c9ae"
server_key_version: "encryptionkey@0"
I20260929 03:06:21.723912 18102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "31a294676da940f1b17af6b92dc2e333"
format_stamp: "Formatted at 2026-09-29 03:06:21 on dist-test-slave-bdps"
server_key: "6203f297615594f1804c568d5c2bc86c"
server_key_iv: "54e54ccf57d338b6693575055e73c9ae"
server_key_version: "encryptionkey@0"
I20260929 03:06:21.727973 18102 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:06:21.730655 18116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:21.731979 18102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 03:06:21.732084 18102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "31a294676da940f1b17af6b92dc2e333"
format_stamp: "Formatted at 2026-09-29 03:06:21 on dist-test-slave-bdps"
server_key: "6203f297615594f1804c568d5c2bc86c"
server_key_iv: "54e54ccf57d338b6693575055e73c9ae"
server_key_version: "encryptionkey@0"
I20260929 03:06:21.732178 18102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:21.737145 18102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:21.737865 18102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:21.738031 18102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:21.738611 18102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:21.739614 18102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:21.739661 18102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:21.739715 18102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:21.739744 18102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:21.748888 18102 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:42977
I20260929 03:06:21.748911 18229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:42977 every 8 connection(s)
I20260929 03:06:21.749874 18102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:06:21.751124 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 18102
I20260929 03:06:21.751245 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:06:21.751614 8005 external_mini_cluster.cc:1428] Setting key 4829d8bd4b7fbedbaa667ca77601e246
I20260929 03:06:21.756651 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:21.751979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:40561 (local address 127.7.209.126:40741)
0929 03:06:21.752160 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:21.752165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:21.753557 (+ 1392us) server_negotiation.cc:407] Connection header received
0929 03:06:21.755062 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:21.755064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:21.755104 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:21.755169 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:21.756290 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:21.756293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:21.756295 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:06:21.756305 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:21.756326 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:06:21.756333 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:21.756403 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:21.756499 (+ 96us) server_negotiation.cc:299] Negotiation successful
0929 03:06:21.756536 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 03:06:21.757632 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:21.752485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:40741 (local address 127.7.209.67:40561)
0929 03:06:21.753395 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:21.753450 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:21.754866 (+ 1416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:21.755321 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:21.755328 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:21.755428 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:21.755998 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:06:21.755999 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:21.756044 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:06:21.756115 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:06:21.756393 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:06:21.756399 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:06:21.756403 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 03:06:21.756537 (+ 134us) client_negotiation.cc:240] Negotiation successful
0929 03:06:21.756752 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":140,"threads_started":1}
I20260929 03:06:21.758631 18230 heartbeater.cc:347] Connected to a master server at 127.7.209.126:40741
I20260929 03:06:21.758857 18230 heartbeater.cc:464] Registering TS with master...
I20260929 03:06:21.759274 18230 heartbeater.cc:511] Master 127.7.209.126:40741 requested a full tablet report, sending...
I20260929 03:06:21.760267 17780 ts_manager.cc:194] Registered new tserver with Master: 31a294676da940f1b17af6b92dc2e333 (127.7.209.67:42977)
I20260929 03:06:21.765362 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 03:06:21.768204 17819 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:06:21.767406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57074 (local address 127.7.209.126:40741)
0929 03:06:21.767538 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:21.767542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:21.767703 (+ 161us) server_negotiation.cc:407] Connection header received
0929 03:06:21.767802 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:21.767804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:21.767835 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:21.767888 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:21.768072 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57074: BlockingRecv error: recv got EOF from 127.0.0.1:57074 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 03:06:21.768335 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 17840
I20260929 03:06:21.774852 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 17971
I20260929 03:06:21.780256 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 18102
I20260929 03:06:21.785255 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 17749
2026-09-29T03:06:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (967 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:06:21 dist-test-slave-bdps krb5kdc[18240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:06:21 dist-test-slave-bdps krb5kdc[18240](info): set up 2 sockets
Sep 29 03:06:21 dist-test-slave-bdps krb5kdc[18240](info): commencing operation
krb5kdc: starting...
W20260929 03:06:22.746809 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:06:22 dist-test-slave-bdps krb5kdc[18240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:06:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:06:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:06:22.875826 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:38165
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:38165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:06:22.998031 18253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:22.998381 18253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:22.998425 18253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:23.002540 18253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:06:23.002590 18253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:23.002609 18253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:23.002624 18253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:06:23.002647 18253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:06:23.008054 18253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:38165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:38165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:23.009254 18253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:23.010468 18253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:06:23.020213 18253 server_base.cc:1034] running on GCE node
W20260929 03:06:23.020146 18259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:23.020123 18261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:23.020126 18258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:23.021142 18253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:23.022584 18253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:23.023751 18253 hybrid_clock.cc:648] HybridClock initialized: now 1790651183023729 us; error 35 us; skew 500 ppm
Sep 29 03:06:23 dist-test-slave-bdps krb5kdc[18240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:06:23.030887 18253 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:06:23.031952 18253 webserver.cc:460] Webserver started at http://127.7.209.126:40401/ using document root <none> and password file <none>
I20260929 03:06:23.032583 18253 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:23.032644 18253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:23.032850 18253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:23.035249 18253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "d652639aeec2430992ade80fb6081eff"
format_stamp: "Formatted at 2026-09-29 03:06:23 on dist-test-slave-bdps"
server_key: "4feef7908c4c3c978a0f88e7d8ad0581"
server_key_iv: "944faa683cf7573bd604fa456e620b13"
server_key_version: "encryptionkey@0"
I20260929 03:06:23.035815 18253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "d652639aeec2430992ade80fb6081eff"
format_stamp: "Formatted at 2026-09-29 03:06:23 on dist-test-slave-bdps"
server_key: "4feef7908c4c3c978a0f88e7d8ad0581"
server_key_iv: "944faa683cf7573bd604fa456e620b13"
server_key_version: "encryptionkey@0"
I20260929 03:06:23.039850 18253 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:06:23.042490 18268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:23.043936 18253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:06:23.044057 18253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "d652639aeec2430992ade80fb6081eff"
format_stamp: "Formatted at 2026-09-29 03:06:23 on dist-test-slave-bdps"
server_key: "4feef7908c4c3c978a0f88e7d8ad0581"
server_key_iv: "944faa683cf7573bd604fa456e620b13"
server_key_version: "encryptionkey@0"
I20260929 03:06:23.044153 18253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:23.069108 18253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:23.073002 18253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:23.073207 18253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:23.081115 18253 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:38165
I20260929 03:06:23.081403 18320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:38165 every 8 connection(s)
I20260929 03:06:23.082310 18253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:06:23.085639 18321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:06:23.091938 18321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff: Bootstrap starting.
I20260929 03:06:23.092582 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 18253
I20260929 03:06:23.092736 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:06:23.093075 8005 external_mini_cluster.cc:1428] Setting key 65c4ddbaa66616bda025a2cdf2872fab
I20260929 03:06:23.095347 18321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff: Neither blocks nor log segments found. Creating new log.
I20260929 03:06:23.096211 18321 log.cc:824] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff: Log is configured to *not* fsync() on all Append() calls
I20260929 03:06:23.098790 18321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff: No bootstrap required, opened a new log
Sep 29 03:06:23 dist-test-slave-bdps krb5kdc[18240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:06:23.103547 18321 raft_consensus.cc:348] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d652639aeec2430992ade80fb6081eff" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38165 } }
I20260929 03:06:23.103837 18321 raft_consensus.cc:374] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:06:23.103905 18321 raft_consensus.cc:729] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d652639aeec2430992ade80fb6081eff, State: Initialized, Role: FOLLOWER
I20260929 03:06:23.104444 18321 consensus_queue.cc:260] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d652639aeec2430992ade80fb6081eff" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38165 } }
I20260929 03:06:23.104589 18321 raft_consensus.cc:388] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:06:23.104660 18321 raft_consensus.cc:482] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:06:23.104768 18321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:23.105846 18321 raft_consensus.cc:504] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d652639aeec2430992ade80fb6081eff" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38165 } }
I20260929 03:06:23.106252 18321 leader_election.cc:302] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d652639aeec2430992ade80fb6081eff; no voters:
I20260929 03:06:23.106652 18321 leader_election.cc:288] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:06:23.106776 18326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:06:23.107020 18326 raft_consensus.cc:686] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [term 1 LEADER]: Becoming Leader. State: Replica: d652639aeec2430992ade80fb6081eff, State: Running, Role: LEADER
I20260929 03:06:23.107398 18326 consensus_queue.cc:237] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d652639aeec2430992ade80fb6081eff" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38165 } }
I20260929 03:06:23.108229 18321 sys_catalog.cc:564] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:06:23.109447 18328 sys_catalog.cc:455] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [sys.catalog]: SysCatalogTable state changed. Reason: New leader d652639aeec2430992ade80fb6081eff. Latest consensus state: current_term: 1 leader_uuid: "d652639aeec2430992ade80fb6081eff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d652639aeec2430992ade80fb6081eff" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38165 } } }
I20260929 03:06:23.109463 18327 sys_catalog.cc:455] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d652639aeec2430992ade80fb6081eff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d652639aeec2430992ade80fb6081eff" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38165 } } }
I20260929 03:06:23.109577 18328 sys_catalog.cc:458] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [sys.catalog]: This master's current role is: LEADER
I20260929 03:06:23.109577 18327 sys_catalog.cc:458] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff [sys.catalog]: This master's current role is: LEADER
I20260929 03:06:23.109716 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.094756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.7.209.126:38165)
0929 03:06:23.095227 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:23.095246 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:23.095298 (+ 52us) server_negotiation.cc:407] Connection header received
0929 03:06:23.096077 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:23.096099 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:23.096573 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:23.097050 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:23.098100 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.099220 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.100409 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.100922 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.101042 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.104091 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:23.104096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:23.104111 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:06:23.104140 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:23.106067 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:23.106544 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:23.106550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:23.106555 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:23.106716 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:23.107063 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:23.107067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:23.107069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:23.107666 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:23.107878 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:23.108166 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 03:06:23.108510 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":133,"threads_started":1}
W20260929 03:06:23.113898 18341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:06:23.113976 18341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:06:23.114051 18335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:06:23.114948 18335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:06:23.120860 18335 catalog_manager.cc:1347] Generated new cluster ID: 03132f8ded7f41219f2b8f1fb1096e0e
I20260929 03:06:23.120929 18335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:06:23.128649 18335 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:06:23.129951 18335 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:06:23.144661 18335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff: Generated new TSK 0
I20260929 03:06:23.145885 18335 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:06:23.181145 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:38165
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:06:23.306952 18347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:23.307327 18347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:23.307397 18347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:23.312183 18347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:23.312276 18347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:23.312372 18347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:06:23.318239 18347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:38165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:23.319566 18347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:23.320822 18347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:23.331634 18353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:23.331657 18355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:23.331635 18352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:23.331877 18347 server_base.cc:1034] running on GCE node
I20260929 03:06:23.332368 18347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:23.333056 18347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:23.334209 18347 hybrid_clock.cc:648] HybridClock initialized: now 1790651183334192 us; error 35 us; skew 500 ppm
Sep 29 03:06:23 dist-test-slave-bdps krb5kdc[18240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:06:23.337198 18347 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:06:23.338286 18347 webserver.cc:460] Webserver started at http://127.7.209.65:35429/ using document root <none> and password file <none>
I20260929 03:06:23.338943 18347 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:23.339013 18347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:23.339250 18347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:23.341923 18347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "40fa365f1596405784d8ba4736ad1354"
format_stamp: "Formatted at 2026-09-29 03:06:23 on dist-test-slave-bdps"
server_key: "3f6110e6ccf9f358cc3f384f13c4244f"
server_key_iv: "acc4009fc87047aee68b4f75ddc173a4"
server_key_version: "encryptionkey@0"
I20260929 03:06:23.342525 18347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "40fa365f1596405784d8ba4736ad1354"
format_stamp: "Formatted at 2026-09-29 03:06:23 on dist-test-slave-bdps"
server_key: "3f6110e6ccf9f358cc3f384f13c4244f"
server_key_iv: "acc4009fc87047aee68b4f75ddc173a4"
server_key_version: "encryptionkey@0"
I20260929 03:06:23.346782 18347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:06:23.349524 18362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:23.350916 18347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:06:23.351027 18347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "40fa365f1596405784d8ba4736ad1354"
format_stamp: "Formatted at 2026-09-29 03:06:23 on dist-test-slave-bdps"
server_key: "3f6110e6ccf9f358cc3f384f13c4244f"
server_key_iv: "acc4009fc87047aee68b4f75ddc173a4"
server_key_version: "encryptionkey@0"
I20260929 03:06:23.351122 18347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:23.382790 18347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:23.386354 18347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:23.386570 18347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:23.387327 18347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:23.388377 18347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:23.388427 18347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:23.388487 18347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:23.388512 18347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:23.398175 18347 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:37797
I20260929 03:06:23.398202 18475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:37797 every 8 connection(s)
I20260929 03:06:23.399261 18347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:06:23.399716 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 18347
I20260929 03:06:23.399852 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:06:23.400211 8005 external_mini_cluster.cc:1428] Setting key 154b3acce6d3d972e615126539ee0e65
Sep 29 03:06:23 dist-test-slave-bdps krb5kdc[18240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
I20260929 03:06:23.414475 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.401393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:36735 (local address 127.7.209.126:38165)
0929 03:06:23.401549 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:23.401554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:23.402725 (+ 1171us) server_negotiation.cc:407] Connection header received
0929 03:06:23.404242 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:23.404246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:23.404317 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:23.404397 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:23.405806 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.406398 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.407676 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.407958 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.408057 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.410675 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:23.410678 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:23.410681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:06:23.410706 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:23.412129 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:23.412702 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:23.412705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:23.412706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:23.412794 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:23.413149 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:23.413153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:23.413155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:23.413400 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:23.413488 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:23.414191 (+ 703us) server_negotiation.cc:299] Negotiation successful
0929 03:06:23.414315 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 03:06:23.415768 18478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.401996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38165 (local address 127.7.209.65:36735)
0929 03:06:23.402539 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:23.402600 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:23.404010 (+ 1410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:23.404615 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:23.404623 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:23.405044 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:23.405649 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.405671 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.406523 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.406527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.407545 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.407555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.408109 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.408113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.408299 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.408921 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:23.408938 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:23.410464 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:06:23.412242 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:06:23.412249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:06:23.412265 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:06:23.412595 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:06:23.412873 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:06:23.412876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:06:23.412879 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:06:23.413029 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:06:23.413499 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:06:23.413505 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:06:23.413912 (+ 407us) client_negotiation.cc:769] Sending connection context
0929 03:06:23.414148 (+ 236us) client_negotiation.cc:240] Negotiation successful
0929 03:06:23.414558 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":132,"threads_started":1}
I20260929 03:06:23.417369 18476 heartbeater.cc:347] Connected to a master server at 127.7.209.126:38165
I20260929 03:06:23.417739 18476 heartbeater.cc:464] Registering TS with master...
I20260929 03:06:23.418457 18476 heartbeater.cc:511] Master 127.7.209.126:38165 requested a full tablet report, sending...
I20260929 03:06:23.420495 18285 ts_manager.cc:194] Registered new tserver with Master: 40fa365f1596405784d8ba4736ad1354 (127.7.209.65:37797)
I20260929 03:06:23.421913 18285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:36735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:06:23.428598 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:38165
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:06:23.550730 18481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:23.551028 18481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:23.551072 18481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:23.555152 18481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:23.555219 18481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:23.555292 18481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:06:23.560686 18481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:38165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:23.561952 18481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:23.563181 18481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:23.574203 18486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:23.574209 18487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:23.574209 18489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:23.574415 18481 server_base.cc:1034] running on GCE node
I20260929 03:06:23.575058 18481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:23.575851 18481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:23.577040 18481 hybrid_clock.cc:648] HybridClock initialized: now 1790651183577017 us; error 40 us; skew 500 ppm
Sep 29 03:06:23 dist-test-slave-bdps krb5kdc[18240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:06:23.580464 18481 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:06:23.581641 18481 webserver.cc:460] Webserver started at http://127.7.209.66:43569/ using document root <none> and password file <none>
I20260929 03:06:23.582346 18481 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:23.582422 18481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:23.582684 18481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:23.585652 18481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "b13259eba2234f349eb5ead883c9ab42"
format_stamp: "Formatted at 2026-09-29 03:06:23 on dist-test-slave-bdps"
server_key: "a9e8b07a9e67dfc66f8915989db96ee5"
server_key_iv: "d88e1d01a3ef88a308734dbddf4bc460"
server_key_version: "encryptionkey@0"
I20260929 03:06:23.586330 18481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "b13259eba2234f349eb5ead883c9ab42"
format_stamp: "Formatted at 2026-09-29 03:06:23 on dist-test-slave-bdps"
server_key: "a9e8b07a9e67dfc66f8915989db96ee5"
server_key_iv: "d88e1d01a3ef88a308734dbddf4bc460"
server_key_version: "encryptionkey@0"
I20260929 03:06:23.591001 18481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:06:23.594035 18496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:23.595616 18481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 03:06:23.595731 18481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "b13259eba2234f349eb5ead883c9ab42"
format_stamp: "Formatted at 2026-09-29 03:06:23 on dist-test-slave-bdps"
server_key: "a9e8b07a9e67dfc66f8915989db96ee5"
server_key_iv: "d88e1d01a3ef88a308734dbddf4bc460"
server_key_version: "encryptionkey@0"
I20260929 03:06:23.595836 18481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:23.614711 18481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:23.618690 18481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:23.618908 18481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:23.619647 18481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:23.620642 18481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:23.620694 18481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:23.620741 18481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:23.620769 18481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:23.630218 18481 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:40905
I20260929 03:06:23.630240 18609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:40905 every 8 connection(s)
I20260929 03:06:23.631227 18481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:06:23.635568 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 18481
I20260929 03:06:23.635671 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:06:23.635984 8005 external_mini_cluster.cc:1428] Setting key 83c29a50b44df5ec45a33fb2b79344cf
Sep 29 03:06:23 dist-test-slave-bdps krb5kdc[18240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:06:23.647125 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.633344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:52555 (local address 127.7.209.126:38165)
0929 03:06:23.633487 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:23.633491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:23.634542 (+ 1051us) server_negotiation.cc:407] Connection header received
0929 03:06:23.636073 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:23.636078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:23.636149 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:23.636262 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:23.637840 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.638618 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.639820 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.640142 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.640234 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.643269 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:23.643274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:23.643277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:06:23.643310 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:23.644977 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:23.645493 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:23.645498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:23.645500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:23.645594 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:23.645919 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:23.645922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:23.645924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:23.646164 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:23.646271 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:23.646796 (+ 525us) server_negotiation.cc:299] Negotiation successful
0929 03:06:23.646933 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:06:23.648134 18612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.633850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38165 (local address 127.7.209.66:52555)
0929 03:06:23.634355 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:23.634403 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:23.635814 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:23.636437 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:23.636445 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:23.636923 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:23.637685 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.637714 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.638738 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.638741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.639697 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.639714 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.640311 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.640314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.640525 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.641213 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:23.641235 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:23.643055 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:06:23.645088 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:06:23.645093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:06:23.645115 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:06:23.645378 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:06:23.645695 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:06:23.645697 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:06:23.645698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:06:23.645827 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:06:23.646253 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:06:23.646258 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:06:23.646508 (+ 250us) client_negotiation.cc:769] Sending connection context
0929 03:06:23.646670 (+ 162us) client_negotiation.cc:240] Negotiation successful
0929 03:06:23.646996 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":129,"threads_started":1}
I20260929 03:06:23.649385 18610 heartbeater.cc:347] Connected to a master server at 127.7.209.126:38165
I20260929 03:06:23.649672 18610 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
I20260929 03:06:23.650369 18610 heartbeater.cc:511] Master 127.7.209.126:38165 requested a full tablet report, sending...
I20260929 03:06:23.651638 18285 ts_manager.cc:194] Registered new tserver with Master: b13259eba2234f349eb5ead883c9ab42 (127.7.209.66:40905)
I20260929 03:06:23.652263 18285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:52555
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:06:23.668211 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:38165
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:06:23.794523 18615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:06:23.794900 18615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:06:23.794955 18615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:06:23.798990 18615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:06:23.799036 18615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:06:23.799091 18615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:06:23.804317 18615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:38165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:06:23.805462 18615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:06:23.806593 18615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:06:23.816948 18620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:23.817062 18615 server_base.cc:1034] running on GCE node
W20260929 03:06:23.816939 18623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:06:23.817268 18621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:06:23.817777 18615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:06:23.818472 18615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:06:23.819623 18615 hybrid_clock.cc:648] HybridClock initialized: now 1790651183819595 us; error 43 us; skew 500 ppm
Sep 29 03:06:23 dist-test-slave-bdps krb5kdc[18240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:06:23.822880 18615 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:06:23.823884 18615 webserver.cc:460] Webserver started at http://127.7.209.67:36123/ using document root <none> and password file <none>
I20260929 03:06:23.824515 18615 fs_manager.cc:359] Metadata directory not provided
I20260929 03:06:23.824585 18615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:06:23.824824 18615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:06:23.827204 18615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "2ea8ba40d9dc48519f942cdacce56ef2"
format_stamp: "Formatted at 2026-09-29 03:06:23 on dist-test-slave-bdps"
server_key: "71b527959f7937e4b288c39fc72b11c8"
server_key_iv: "dcca55d215a46a8c72bb8244bd4488d5"
server_key_version: "encryptionkey@0"
I20260929 03:06:23.827816 18615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "2ea8ba40d9dc48519f942cdacce56ef2"
format_stamp: "Formatted at 2026-09-29 03:06:23 on dist-test-slave-bdps"
server_key: "71b527959f7937e4b288c39fc72b11c8"
server_key_iv: "dcca55d215a46a8c72bb8244bd4488d5"
server_key_version: "encryptionkey@0"
I20260929 03:06:23.831853 18615 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:06:23.834357 18630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:23.835692 18615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 03:06:23.835803 18615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "2ea8ba40d9dc48519f942cdacce56ef2"
format_stamp: "Formatted at 2026-09-29 03:06:23 on dist-test-slave-bdps"
server_key: "71b527959f7937e4b288c39fc72b11c8"
server_key_iv: "dcca55d215a46a8c72bb8244bd4488d5"
server_key_version: "encryptionkey@0"
I20260929 03:06:23.835891 18615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:06:23.865670 18615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:06:23.869117 18615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:06:23.869303 18615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:06:23.870277 18615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:06:23.871323 18615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:06:23.871373 18615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:23.871416 18615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:06:23.871435 18615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:06:23.881321 18615 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:37991
I20260929 03:06:23.881348 18743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:37991 every 8 connection(s)
I20260929 03:06:23.882616 18615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:06:23.886452 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 18615
I20260929 03:06:23.886613 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:06:23.887027 8005 external_mini_cluster.cc:1428] Setting key 5b9f0dbfb5531dce98a2e9b5ed013be2
Sep 29 03:06:23 dist-test-slave-bdps krb5kdc[18240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651183, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:06:23.898165 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.884909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:58783 (local address 127.7.209.126:38165)
0929 03:06:23.885086 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:23.885091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:23.886322 (+ 1231us) server_negotiation.cc:407] Connection header received
0929 03:06:23.888184 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:23.888187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:23.888247 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:23.888322 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:23.889947 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.890543 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.891743 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.891956 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.892025 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.894744 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:23.894748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:23.894750 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:06:23.894779 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:23.896046 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:23.896529 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:23.896531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:23.896533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:23.896666 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:23.896956 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:23.896958 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:23.896960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:23.897162 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:23.897243 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:23.897854 (+ 611us) server_negotiation.cc:299] Negotiation successful
0929 03:06:23.897992 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 03:06:23.899250 18746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.885467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38165 (local address 127.7.209.67:58783)
0929 03:06:23.886065 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:23.886138 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:23.887936 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:23.888564 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:23.888575 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:23.889047 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:06:23.889805 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.889823 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.890631 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.890633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.891651 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.891659 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.892073 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.892076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.892258 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.892929 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:06:23.892947 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:06:23.894574 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:06:23.896141 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:06:23.896147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:06:23.896163 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:06:23.896451 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:06:23.896736 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:06:23.896738 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:06:23.896739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:06:23.896883 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:06:23.897241 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:06:23.897247 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:06:23.897543 (+ 296us) client_negotiation.cc:769] Sending connection context
0929 03:06:23.897767 (+ 224us) client_negotiation.cc:240] Negotiation successful
0929 03:06:23.898115 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":157,"threads_started":1}
I20260929 03:06:23.900732 18744 heartbeater.cc:347] Connected to a master server at 127.7.209.126:38165
I20260929 03:06:23.901074 18744 heartbeater.cc:464] Registering TS with master...
I20260929 03:06:23.901809 18744 heartbeater.cc:511] Master 127.7.209.126:38165 requested a full tablet report, sending...
I20260929 03:06:23.902954 18285 ts_manager.cc:194] Registered new tserver with Master: 2ea8ba40d9dc48519f942cdacce56ef2 (127.7.209.67:37991)
I20260929 03:06:23.903591 18285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:58783
I20260929 03:06:23.913151 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:06:23.922780 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.915356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45030 (local address 127.7.209.126:38165)
0929 03:06:23.915480 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:23.915485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:23.915698 (+ 213us) server_negotiation.cc:407] Connection header received
0929 03:06:23.915806 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:23.915809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:23.915855 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:23.915921 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:06:23.916605 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.917173 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.918077 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.918309 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.918388 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.919852 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:06:23.919856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:06:23.919858 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:06:23.919881 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:06:23.921286 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:23.921716 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:23.921719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:23.921720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:23.921797 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:06:23.922002 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:06:23.922004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:06:23.922006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:06:23.922251 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:06:23.922358 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:23.922542 (+ 184us) server_negotiation.cc:299] Negotiation successful
0929 03:06:23.922642 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 03:06:23.925798 18285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:06:23.929358 18285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:06:23.944958 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.938590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:37797 (local address 127.0.0.1:54852)
0929 03:06:23.939504 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:23.939717 (+ 213us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:23.939799 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:23.940418 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:23.940424 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:23.940450 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:06:23.940610 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.940616 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.942273 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.942276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.943354 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.943364 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.944341 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.944345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.944456 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.944502 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 03:06:23.944649 (+ 147us) client_negotiation.cc:240] Negotiation successful
0929 03:06:23.944784 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":67,"thread_start_us":129,"threads_started":1}
I20260929 03:06:23.945286 18759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.939522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:37991 (local address 127.0.0.1:39382)
0929 03:06:23.939882 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:23.939895 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:23.939993 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:23.940780 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:23.940783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:23.940801 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:06:23.940882 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.940886 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.942299 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.942302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.943796 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.943806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.944916 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.944920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.945010 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.945034 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 03:06:23.945078 (+ 44us) client_negotiation.cc:240] Negotiation successful
0929 03:06:23.945150 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":144,"thread_start_us":86,"threads_started":1}
I20260929 03:06:23.945710 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.938754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.7.209.65:37797)
0929 03:06:23.939272 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:23.939280 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:23.939882 (+ 602us) server_negotiation.cc:407] Connection header received
0929 03:06:23.939984 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:23.939991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:23.940253 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:23.940433 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:06:23.940831 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.942148 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.943487 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.944212 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.944323 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.944502 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:23.945353 (+ 851us) server_negotiation.cc:299] Negotiation successful
0929 03:06:23.945492 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":75,"threads_started":1}
I20260929 03:06:23.946064 18760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.939936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39382 (local address 127.7.209.67:37991)
0929 03:06:23.940269 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:23.940291 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:23.940314 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:06:23.940396 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:23.940402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:23.940617 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:23.941008 (+ 391us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:06:23.941201 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.942185 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.943920 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.944778 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.945564 (+ 786us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.945760 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:23.945814 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 03:06:23.945915 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":124,"threads_started":1}
I20260929 03:06:23.946466 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.939440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36674 (local address 127.7.209.66:40905)
0929 03:06:23.940060 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:23.940072 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:23.940093 (+ 21us) server_negotiation.cc:407] Connection header received
0929 03:06:23.940166 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:23.940171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:23.940317 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:23.940446 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:06:23.940867 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.942147 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.944577 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.945477 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.945580 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.945760 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:23.946226 (+ 466us) server_negotiation.cc:299] Negotiation successful
0929 03:06:23.946325 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":131,"threads_started":1}
I20260929 03:06:23.946585 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.938899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:40905 (local address 127.0.0.1:36674)
0929 03:06:23.939504 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:23.939562 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:23.939705 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:23.940621 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:23.940623 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:23.940639 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:06:23.940734 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.940741 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.942297 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.942300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.943795 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.943806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.945596 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.945626 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.945712 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.945736 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 03:06:23.945917 (+ 181us) client_negotiation.cc:240] Negotiation successful
0929 03:06:23.945990 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":198,"thread_start_us":109,"threads_started":1}
I20260929 03:06:23.949280 18410 tablet_service.cc:1467] Processing CreateTablet for tablet 4036ffd151634def9f58eb9b47922d86 (DEFAULT_TABLE table=test-table [id=f0e8e452eeaa4a4c84234e116494b57c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:06:23.949282 18678 tablet_service.cc:1467] Processing CreateTablet for tablet 4036ffd151634def9f58eb9b47922d86 (DEFAULT_TABLE table=test-table [id=f0e8e452eeaa4a4c84234e116494b57c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:06:23.949752 18544 tablet_service.cc:1467] Processing CreateTablet for tablet 4036ffd151634def9f58eb9b47922d86 (DEFAULT_TABLE table=test-table [id=f0e8e452eeaa4a4c84234e116494b57c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:06:23.950469 18410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4036ffd151634def9f58eb9b47922d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:06:23.950486 18678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4036ffd151634def9f58eb9b47922d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:06:23.950632 18544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4036ffd151634def9f58eb9b47922d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:06:23.957453 18761 tablet_bootstrap.cc:492] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354: Bootstrap starting.
I20260929 03:06:23.958997 18762 tablet_bootstrap.cc:492] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42: Bootstrap starting.
I20260929 03:06:23.960692 18761 tablet_bootstrap.cc:654] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354: Neither blocks nor log segments found. Creating new log.
I20260929 03:06:23.961721 18761 log.cc:824] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354: Log is configured to *not* fsync() on all Append() calls
I20260929 03:06:23.961936 18762 tablet_bootstrap.cc:654] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42: Neither blocks nor log segments found. Creating new log.
I20260929 03:06:23.961956 18763 tablet_bootstrap.cc:492] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2: Bootstrap starting.
I20260929 03:06:23.962728 18762 log.cc:824] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42: Log is configured to *not* fsync() on all Append() calls
I20260929 03:06:23.964035 18761 tablet_bootstrap.cc:492] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354: No bootstrap required, opened a new log
I20260929 03:06:23.964052 18763 tablet_bootstrap.cc:654] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2: Neither blocks nor log segments found. Creating new log.
I20260929 03:06:23.964262 18761 ts_tablet_manager.cc:1397] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 03:06:23.964820 18762 tablet_bootstrap.cc:492] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42: No bootstrap required, opened a new log
I20260929 03:06:23.965011 18762 ts_tablet_manager.cc:1397] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260929 03:06:23.965138 18763 log.cc:824] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2: Log is configured to *not* fsync() on all Append() calls
I20260929 03:06:23.966789 18763 tablet_bootstrap.cc:492] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2: No bootstrap required, opened a new log
I20260929 03:06:23.966962 18763 ts_tablet_manager.cc:1397] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 03:06:23.969141 18762 raft_consensus.cc:348] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa365f1596405784d8ba4736ad1354" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37797 } } peers { permanent_uuid: "2ea8ba40d9dc48519f942cdacce56ef2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37991 } } peers { permanent_uuid: "b13259eba2234f349eb5ead883c9ab42" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40905 } }
I20260929 03:06:23.969141 18761 raft_consensus.cc:348] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa365f1596405784d8ba4736ad1354" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37797 } } peers { permanent_uuid: "2ea8ba40d9dc48519f942cdacce56ef2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37991 } } peers { permanent_uuid: "b13259eba2234f349eb5ead883c9ab42" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40905 } }
I20260929 03:06:23.969440 18761 raft_consensus.cc:374] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:06:23.969439 18762 raft_consensus.cc:374] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:06:23.969498 18761 raft_consensus.cc:729] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40fa365f1596405784d8ba4736ad1354, State: Initialized, Role: FOLLOWER
I20260929 03:06:23.969498 18762 raft_consensus.cc:729] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13259eba2234f349eb5ead883c9ab42, State: Initialized, Role: FOLLOWER
I20260929 03:06:23.969965 18763 raft_consensus.cc:348] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa365f1596405784d8ba4736ad1354" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37797 } } peers { permanent_uuid: "2ea8ba40d9dc48519f942cdacce56ef2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37991 } } peers { permanent_uuid: "b13259eba2234f349eb5ead883c9ab42" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40905 } }
I20260929 03:06:23.970082 18762 consensus_queue.cc:260] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa365f1596405784d8ba4736ad1354" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37797 } } peers { permanent_uuid: "2ea8ba40d9dc48519f942cdacce56ef2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37991 } } peers { permanent_uuid: "b13259eba2234f349eb5ead883c9ab42" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40905 } }
I20260929 03:06:23.970082 18761 consensus_queue.cc:260] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa365f1596405784d8ba4736ad1354" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37797 } } peers { permanent_uuid: "2ea8ba40d9dc48519f942cdacce56ef2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37991 } } peers { permanent_uuid: "b13259eba2234f349eb5ead883c9ab42" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40905 } }
I20260929 03:06:23.970180 18763 raft_consensus.cc:374] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:06:23.970235 18763 raft_consensus.cc:729] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea8ba40d9dc48519f942cdacce56ef2, State: Initialized, Role: FOLLOWER
I20260929 03:06:23.970705 18763 consensus_queue.cc:260] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa365f1596405784d8ba4736ad1354" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37797 } } peers { permanent_uuid: "2ea8ba40d9dc48519f942cdacce56ef2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37991 } } peers { permanent_uuid: "b13259eba2234f349eb5ead883c9ab42" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40905 } }
I20260929 03:06:23.970976 18610 heartbeater.cc:503] Master 127.7.209.126:38165 was elected leader, sending a full tablet report...
I20260929 03:06:23.971122 18476 heartbeater.cc:503] Master 127.7.209.126:38165 was elected leader, sending a full tablet report...
I20260929 03:06:23.971612 18763 ts_tablet_manager.cc:1428] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260929 03:06:23.971937 18744 heartbeater.cc:503] Master 127.7.209.126:38165 was elected leader, sending a full tablet report...
I20260929 03:06:23.972709 18762 ts_tablet_manager.cc:1428] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260929 03:06:23.974928 18761 ts_tablet_manager.cc:1428] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260929 03:06:23.975358 18767 raft_consensus.cc:482] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:06:23.975647 18767 raft_consensus.cc:504] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa365f1596405784d8ba4736ad1354" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37797 } } peers { permanent_uuid: "2ea8ba40d9dc48519f942cdacce56ef2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37991 } } peers { permanent_uuid: "b13259eba2234f349eb5ead883c9ab42" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40905 } }
I20260929 03:06:23.977021 18767 leader_election.cc:288] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40fa365f1596405784d8ba4736ad1354 (127.7.209.65:37797), 2ea8ba40d9dc48519f942cdacce56ef2 (127.7.209.67:37991)
I20260929 03:06:23.981426 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.977370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:56305 (local address 127.7.209.65:37797)
0929 03:06:23.977747 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:23.977752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:23.977766 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:06:23.977813 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:23.977816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:23.977864 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:23.977944 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:06:23.978272 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.979040 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.980274 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.980923 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.981014 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.981062 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:23.981249 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 03:06:23.981314 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260929 03:06:23.981447 18612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.977283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:37797 (local address 127.7.209.66:56305)
0929 03:06:23.977434 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:23.977451 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:23.977543 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:23.977962 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:23.977966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:23.977989 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:06:23.978149 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.978158 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.979138 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.979141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.980158 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.980166 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.981022 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.981026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.981134 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.981160 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 03:06:23.981234 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 03:06:23.981316 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 03:06:23.982089 18760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.977701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:48315 (local address 127.7.209.67:37991)
0929 03:06:23.978136 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:23.978142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:23.978165 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:06:23.978352 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:23.978360 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:23.978437 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:23.978514 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:06:23.978950 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.979654 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.980881 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.981449 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.981558 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.981629 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:23.981933 (+ 304us) server_negotiation.cc:299] Negotiation successful
0929 03:06:23.981994 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345}
I20260929 03:06:23.982120 18770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:23.977559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:37991 (local address 127.7.209.66:48315)
0929 03:06:23.978031 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:23.978073 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 03:06:23.978188 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:06:23.978704 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:06:23.978707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:06:23.978723 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:06:23.978845 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.978851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.979766 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.979769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.980768 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:06:23.980778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:06:23.981705 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:06:23.981708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:06:23.981806 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:23.981832 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 03:06:23.981912 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 03:06:23.981985 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":205,"threads_started":1}
I20260929 03:06:23.982163 18430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4036ffd151634def9f58eb9b47922d86" candidate_uuid: "b13259eba2234f349eb5ead883c9ab42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40fa365f1596405784d8ba4736ad1354" is_pre_election: true
I20260929 03:06:23.982532 18430 raft_consensus.cc:2454] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b13259eba2234f349eb5ead883c9ab42 in term 0.
I20260929 03:06:23.982724 18698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4036ffd151634def9f58eb9b47922d86" candidate_uuid: "b13259eba2234f349eb5ead883c9ab42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea8ba40d9dc48519f942cdacce56ef2" is_pre_election: true
I20260929 03:06:23.983034 18698 raft_consensus.cc:2454] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b13259eba2234f349eb5ead883c9ab42 in term 0.
I20260929 03:06:23.983057 18500 leader_election.cc:302] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40fa365f1596405784d8ba4736ad1354, b13259eba2234f349eb5ead883c9ab42; no voters:
I20260929 03:06:23.983325 18767 raft_consensus.cc:2781] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:06:23.983408 18767 raft_consensus.cc:482] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:06:23.983440 18767 raft_consensus.cc:3037] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:23.984827 18767 raft_consensus.cc:504] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa365f1596405784d8ba4736ad1354" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37797 } } peers { permanent_uuid: "2ea8ba40d9dc48519f942cdacce56ef2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37991 } } peers { permanent_uuid: "b13259eba2234f349eb5ead883c9ab42" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40905 } }
I20260929 03:06:23.985159 18767 leader_election.cc:288] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [CANDIDATE]: Term 1 election: Requested vote from peers 40fa365f1596405784d8ba4736ad1354 (127.7.209.65:37797), 2ea8ba40d9dc48519f942cdacce56ef2 (127.7.209.67:37991)
I20260929 03:06:23.985515 18698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4036ffd151634def9f58eb9b47922d86" candidate_uuid: "b13259eba2234f349eb5ead883c9ab42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea8ba40d9dc48519f942cdacce56ef2"
I20260929 03:06:23.985505 18430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4036ffd151634def9f58eb9b47922d86" candidate_uuid: "b13259eba2234f349eb5ead883c9ab42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40fa365f1596405784d8ba4736ad1354"
I20260929 03:06:23.985636 18698 raft_consensus.cc:3037] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:23.985653 18430 raft_consensus.cc:3037] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:06:23.986724 18698 raft_consensus.cc:2454] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13259eba2234f349eb5ead883c9ab42 in term 1.
I20260929 03:06:23.986816 18430 raft_consensus.cc:2454] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b13259eba2234f349eb5ead883c9ab42 in term 1.
I20260929 03:06:23.986958 18500 leader_election.cc:302] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea8ba40d9dc48519f942cdacce56ef2, b13259eba2234f349eb5ead883c9ab42; no voters:
I20260929 03:06:23.987131 18767 raft_consensus.cc:2781] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:06:23.987365 18767 raft_consensus.cc:686] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 1 LEADER]: Becoming Leader. State: Replica: b13259eba2234f349eb5ead883c9ab42, State: Running, Role: LEADER
I20260929 03:06:23.987730 18767 consensus_queue.cc:237] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa365f1596405784d8ba4736ad1354" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37797 } } peers { permanent_uuid: "2ea8ba40d9dc48519f942cdacce56ef2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37991 } } peers { permanent_uuid: "b13259eba2234f349eb5ead883c9ab42" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40905 } }
I20260929 03:06:23.991223 18285 catalog_manager.cc:5495] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 reported cstate change: term changed from 0 to 1, leader changed from <none> to b13259eba2234f349eb5ead883c9ab42 (127.7.209.66). New cstate: current_term: 1 leader_uuid: "b13259eba2234f349eb5ead883c9ab42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40fa365f1596405784d8ba4736ad1354" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ea8ba40d9dc48519f942cdacce56ef2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b13259eba2234f349eb5ead883c9ab42" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40905 } health_report { overall_health: HEALTHY } } }
I20260929 03:06:24.005342 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:24.002009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36678 (local address 127.7.209.66:40905)
0929 03:06:24.002128 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 03:06:24.002131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:06:24.002145 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:06:24.002295 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:06:24.002297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:06:24.002353 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:06:24.002446 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:06:24.002664 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:24.003172 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:24.004000 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:06:24.004213 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:06:24.004280 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:06:24.004514 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:06:24.005016 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:06:24.005109 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 03:06:24.005134 (+ 25us) server_negotiation.cc:299] Negotiation successful
0929 03:06:24.005223 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:06:24.012161 18698 raft_consensus.cc:1261] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2 [term 1 FOLLOWER]: Refusing update from remote peer b13259eba2234f349eb5ead883c9ab42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:06:24.012161 18430 raft_consensus.cc:1261] T 4036ffd151634def9f58eb9b47922d86 P 40fa365f1596405784d8ba4736ad1354 [term 1 FOLLOWER]: Refusing update from remote peer b13259eba2234f349eb5ead883c9ab42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:06:24.012944 18771 consensus_queue.cc:1035] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea8ba40d9dc48519f942cdacce56ef2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 37991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:06:24.013149 18767 consensus_queue.cc:1035] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40fa365f1596405784d8ba4736ad1354" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:06:24.022763 18776 mvcc.cc:205] Tried to move back new op lower bound from 7334507249705832448 to 7334507249616535552. Current Snapshot: MvccSnapshot[applied={T|T < 7334507249705832448}]
I20260929 03:06:24.033172 18282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:06:24.033387 18282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:06:24.035817 18282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d652639aeec2430992ade80fb6081eff: Sending DeleteTablet for 3 replicas of tablet 4036ffd151634def9f58eb9b47922d86
I20260929 03:06:24.036659 18410 tablet_service.cc:1514] Processing DeleteTablet for tablet 4036ffd151634def9f58eb9b47922d86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:06:24 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:54852
I20260929 03:06:24.036659 18544 tablet_service.cc:1514] Processing DeleteTablet for tablet 4036ffd151634def9f58eb9b47922d86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:06:24 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:36674
I20260929 03:06:24.037179 18784 tablet_replica.cc:330] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42: stopping tablet replica
I20260929 03:06:24.037197 18678 tablet_service.cc:1514] Processing DeleteTablet for tablet 4036ffd151634def9f58eb9b47922d86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:06:24 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:39382
I20260929 03:06:24.037659 18784 raft_consensus.cc:2229] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:06:24.037690 18786 tablet_replica.cc:330] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2: stopping tablet replica
I20260929 03:06:24.038049 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 18347
I20260929 03:06:24.038080 18786 raft_consensus.cc:2229] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:06:24.038889 18786 raft_consensus.cc:2258] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:06:24.040823 18784 raft_consensus.cc:2258] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:06:24.042421 18784 ts_tablet_manager.cc:1905] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:06:24.045408 18784 ts_tablet_manager.cc:1918] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:06:24.045501 18784 log.cc:1196] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/4036ffd151634def9f58eb9b47922d86
I20260929 03:06:24.045831 18784 ts_tablet_manager.cc:1939] T 4036ffd151634def9f58eb9b47922d86 P b13259eba2234f349eb5ead883c9ab42: Deleting consensus metadata
I20260929 03:06:24.046077 18786 ts_tablet_manager.cc:1905] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:06:24.047024 18270 catalog_manager.cc:4841] TS b13259eba2234f349eb5ead883c9ab42 (127.7.209.66:40905): tablet 4036ffd151634def9f58eb9b47922d86 (table test-table [id=f0e8e452eeaa4a4c84234e116494b57c]) successfully deleted
W20260929 03:06:24.047149 18272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:37797 (error 108)
I20260929 03:06:24.048115 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 18481
I20260929 03:06:24.048264 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:06:24.047911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:37797 (local address 127.0.0.1:54858)
0929 03:06:24.048086 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 03:06:24.048157 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:37797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260929 03:06:24.048786 18272 catalog_manager.cc:4584] TS 40fa365f1596405784d8ba4736ad1354 (127.7.209.65:37797): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4036ffd151634def9f58eb9b47922d86: Network error: Client connection negotiation failed: client connection to 127.7.209.65:37797: connect: Connection refused (error 111)
I20260929 03:06:24.050932 18786 ts_tablet_manager.cc:1918] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:06:24.051034 18786 log.cc:1196] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/4036ffd151634def9f58eb9b47922d86
I20260929 03:06:24.051388 18786 ts_tablet_manager.cc:1939] T 4036ffd151634def9f58eb9b47922d86 P 2ea8ba40d9dc48519f942cdacce56ef2: Deleting consensus metadata
I20260929 03:06:24.052683 18272 catalog_manager.cc:4841] TS 2ea8ba40d9dc48519f942cdacce56ef2 (127.7.209.67:37991): tablet 4036ffd151634def9f58eb9b47922d86 (table test-table [id=f0e8e452eeaa4a4c84234e116494b57c]) successfully deleted
I20260929 03:06:24.054904 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 18615
I20260929 03:06:24.060238 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 18253
2026-09-29T03:06:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2276 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:06:24 dist-test-slave-bdps krb5kdc[18790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:06:24 dist-test-slave-bdps krb5kdc[18790](info): set up 2 sockets
Sep 29 03:06:24 dist-test-slave-bdps krb5kdc[18790](info): commencing operation
krb5kdc: starting...
W20260929 03:06:25.014868 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:06:25 dist-test-slave-bdps krb5kdc[18790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:06:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:06:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.209.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.209.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/rangeradmin_127.7.209.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/rangeradmin_127.7.209.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.209.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.209.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/rangerlookup_127.7.209.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/rangerlookup_127.7.209.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.209.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.209.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/HTTP_127.7.209.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/HTTP_127.7.209.84@KRBTEST.COM.keytab.
I20260929 03:06:25.188613 8005 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskC1pyL7/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/postgres -l logfile start
2026-09-29 03:06:27.489 UTC [18820] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 03:06:27.489 UTC [18820] LOG: listening on IPv4 address "127.7.209.84", port 47409
2026-09-29 03:06:27.492 UTC [18820] LOG: listening on Unix socket "/tmp/.s.PGSQL.47409"
2026-09-29 03:06:27.522 UTC [18823] LOG: database system was shut down at 2026-09-29 03:06:26 UTC
2026-09-29 03:06:27.526 UTC [18820] LOG: database system is ready to accept connections
I20260929 03:06:28.398751 8005 mini_postgres.cc:96] Postgres bound to 47409
2026-09-29 03:06:28.406 UTC [18831] FATAL: database "slave" does not exist
127.7.209.84:47409 - accepting connections
I20260929 03:06:28.407752 8005 mini_ranger.cc:161] Starting Ranger...
I20260929 03:06:28.425757 8005 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 03:06:28.842362 8005 mini_ranger.cc:87] Created ranger Postgres database
I20260929 03:06:28.842469 8005 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-admin
2026-09-29 03:06:29,138 [I] DB FLAVOR :POSTGRES
2026-09-29 03:06:29,138 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 03:06:29,138 [I] Checking connection..
2026-09-29 03:06:29,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 03:06:29,414 [I] Checking connection passed.
2026-09-29 03:06:29,414 [I] --------- Verifying version history table ---------
2026-09-29 03:06:29,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 03:06:29,688 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 03:06:29,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 03:06:29,957 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 03:06:29,957 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 03:06:29,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 03:06:30.227 UTC [18895] WARNING: there is no transaction in progress
2026-09-29 03:06:30,233 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 03:06:30,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 03:06:30,485 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 03:06:30,486 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 03:06:30,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 03:06:30,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 03:06:31,005 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-bdps.c.gcp-upstream.internal','N') ;"
2026-09-29 03:06:31,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 03:06:31,519 [I] Table x_portal_user does not exist in database ranger
2026-09-29 03:06:31,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 03:06:31,789 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 03:06:31,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 03:06:32,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 03:06:32,326 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 03:06:32,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 03:06:33.258 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.274 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.289 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.304 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.323 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.339 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.397 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.403 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.417 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.427 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.437 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.450 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.458 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.464 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.471 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:33.477 UTC [19003] WARNING: there is no transaction in progress
2026-09-29 03:06:34,051 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 03:06:34,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 03:06:34,293 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 03:06:34,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 03:06:34,547 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 03:06:34,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 03:06:34,817 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 03:06:34,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 03:06:35,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 03:06:35,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-bdps.c.gcp-upstream.internal';"
2026-09-29 03:06:35,616 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 03:06:35,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 03:06:35,890 [I] DB_PATCHES have already been applied
I20260929 03:06:35.896978 8005 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-admin:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskC1pyL7/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 03:06:35.897058 8005 mini_ranger.cc:193] Using host: 127.7.209.84
I20260929 03:06:35.901598 8005 mini_ranger.cc:227] Ranger admin URL: http://127.7.209.84:53809
Sep 29, 2026 3:06:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 3:06:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 3:06:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 3:06:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 3:06:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 3:06:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 3:06:36 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 03:06:37.978320 8005 mini_ranger.cc:160] Time spent starting Ranger: real 9.571s user 0.010s sys 0.006s
Sep 29, 2026 3:06:58 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 3:06:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 3:06:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 3:06:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 3:06:58 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 3:06:59 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 03:07:00.280321 8005 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.209.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.209.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/rangerkms_127.7.209.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/rangerkms_127.7.209.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.209.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.209.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.209.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/HTTP_127.7.209.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.209.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/HTTP_127.7.209.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 03:07:00 dist-test-slave-bdps krb5kdc[18790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651220, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 03:07:00.396798 8005 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 03:07:00.414170 8005 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 03:07:01.238585 8005 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 03:07:01.238777 8005 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-kms
I20260929 03:07:01.238932 8005 mini_ranger_kms.cc:227] Using postgres at 127.7.209.84:47409
2026-09-29 03:07:01,902 [I] DB FLAVOR :POSTGRES
2026-09-29 03:07:01,902 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 03:07:01,903 [I] Checking connection
2026-09-29 03:07:01,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 03:07:02,233 [I] connection success
2026-09-29 03:07:02,234 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 03:07:02,234 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 03:07:02,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 03:07:02,489 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 03:07:02,490 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 03:07:02,490 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 03:07:02,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:47409/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 03:07:02,802 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 03:07:03.354238 8005 mini_ranger_kms.cc:310] Created kms service
I20260929 03:07:03.583650 8005 mini_ranger_kms.cc:326] Created kudu user
I20260929 03:07:03.634410 8005 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 03:07:03 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651220, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:07:04.047079 8005 mini_ranger_kms.cc:384] Added ranger policy
I20260929 03:07:04.047276 8005 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-kms:/tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 03:07:04.047302 8005 mini_ranger_kms.cc:240] Using host: 127.7.209.84
I20260929 03:07:04.052469 8005 mini_ranger_kms.cc:276] Ranger KMS PID: 19185
I20260929 03:07:04.052652 8005 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.209.84:45325
Sep 29, 2026 3:07:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 3:07:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 3:07:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 3:07:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 3:07:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 3:07:05 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 03:07:06.056058 8005 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.659s user 0.011s sys 0.627s
I20260929 03:07:06.056218 8005 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 03:07:06.056366 8005 mini_ranger_kms.cc:401] 127.7.209.84:45325/kms/v1/keys
Sep 29 03:07:06 dist-test-slave-bdps krb5kdc[18790](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790651226, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.209.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 03:07:07 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651226, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.209.84@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
Sep 29 03:07:07 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651226, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.209.84@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
Sep 29 03:07:10 dist-test-slave-bdps krb5kdc[18790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:07:10.076107 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:36727
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:36727
--ranger_config_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.209.84:45325/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:07:10.248553 19235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:07:10.248899 19235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:07:10.248943 19235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:07:10.253166 19235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:07:10.253233 19235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:07:10.253252 19235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:07:10.253268 19235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:07:10.253281 19235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:07:10.259212 19235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.209.84:45325/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:36727
--ranger_config_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:36727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:07:10.260519 19235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:07:10.261854 19235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:07:10.271670 19240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:07:10.271832 19243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:07:10.271658 19241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:07:10.272220 19235 server_base.cc:1034] running on GCE node
I20260929 03:07:10.272853 19235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:07:10.274214 19235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:07:10.275373 19235 hybrid_clock.cc:648] HybridClock initialized: now 1790651230275349 us; error 45 us; skew 500 ppm
Sep 29 03:07:10 dist-test-slave-bdps krb5kdc[18790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651230, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:07:10.278523 19235 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:07:10.279688 19235 webserver.cc:460] Webserver started at http://127.7.209.126:42731/ using document root <none> and password file <none>
I20260929 03:07:10.280385 19235 fs_manager.cc:359] Metadata directory not provided
I20260929 03:07:10.280447 19235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:07:10.280707 19235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:07:10 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651230, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:07:10.344262 19235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "05eb3696e5724d4e80acac3e1b99a0b8"
format_stamp: "Formatted at 2026-09-29 03:07:10 on dist-test-slave-bdps"
server_key: "0002513ea2019adb897531ad99bd2bb6"
server_key_iv: "26d1a3d55a619ec4e2aab5a014556d2e"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:10.345108 19235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "05eb3696e5724d4e80acac3e1b99a0b8"
format_stamp: "Formatted at 2026-09-29 03:07:10 on dist-test-slave-bdps"
server_key: "0002513ea2019adb897531ad99bd2bb6"
server_key_iv: "26d1a3d55a619ec4e2aab5a014556d2e"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:10.349901 19235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 03:07:10.367069 19250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:10.368876 19235 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 03:07:10.369030 19235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "05eb3696e5724d4e80acac3e1b99a0b8"
format_stamp: "Formatted at 2026-09-29 03:07:10 on dist-test-slave-bdps"
server_key: "0002513ea2019adb897531ad99bd2bb6"
server_key_iv: "26d1a3d55a619ec4e2aab5a014556d2e"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:10.369207 19235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:07:10.388381 19235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:07:10.392130 19235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:07:10.392387 19235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:07:10.407960 19235 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:36727
I20260929 03:07:10.408023 19302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:36727 every 8 connection(s)
I20260929 03:07:10.409349 19235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:07:10.413323 19303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:07:10.416484 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 19235
I20260929 03:07:10.416646 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:07:10.417001 8005 external_mini_cluster.cc:1428] Setting key 2a287b14882bb0f1a35f1b87b397019c
I20260929 03:07:10.421660 19303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8: Bootstrap starting.
I20260929 03:07:10.425199 19303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8: Neither blocks nor log segments found. Creating new log.
I20260929 03:07:10.426673 19303 log.cc:824] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8: Log is configured to *not* fsync() on all Append() calls
I20260929 03:07:10.429191 19303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8: No bootstrap required, opened a new log
Sep 29 03:07:10 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:07:10.434652 19303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05eb3696e5724d4e80acac3e1b99a0b8" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 36727 } }
I20260929 03:07:10.434980 19303 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:07:10.435060 19303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05eb3696e5724d4e80acac3e1b99a0b8, State: Initialized, Role: FOLLOWER
I20260929 03:07:10.435627 19303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05eb3696e5724d4e80acac3e1b99a0b8" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 36727 } }
I20260929 03:07:10.435779 19303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:07:10.435860 19303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:07:10.435981 19306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:10.418932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37456 (local address 127.7.209.126:36727)
0929 03:07:10.419530 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:10.419554 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:10.419610 (+ 56us) server_negotiation.cc:407] Connection header received
0929 03:07:10.420593 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:10.420624 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:10.421159 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:10.421635 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:07:10.422575 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:10.424281 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:10.425520 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:10.426336 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:10.426517 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:10.429972 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:07:10.429980 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:07:10.429999 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:07:10.430033 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:07:10.432240 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:10.432833 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:10.432840 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:10.432848 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:10.432999 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:10.433330 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:10.433333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:10.433335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:10.433976 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:07:10.434222 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:10.434499 (+ 277us) server_negotiation.cc:299] Negotiation successful
0929 03:07:10.434835 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":178,"threads_started":1}
I20260929 03:07:10.435987 19303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:07:10.437179 19303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05eb3696e5724d4e80acac3e1b99a0b8" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 36727 } }
I20260929 03:07:10.437654 19303 leader_election.cc:302] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05eb3696e5724d4e80acac3e1b99a0b8; no voters:
I20260929 03:07:10.438051 19303 leader_election.cc:288] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:07:10.438185 19308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:07:10.438426 19308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [term 1 LEADER]: Becoming Leader. State: Replica: 05eb3696e5724d4e80acac3e1b99a0b8, State: Running, Role: LEADER
I20260929 03:07:10.439486 19308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05eb3696e5724d4e80acac3e1b99a0b8" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 36727 } }
I20260929 03:07:10.439543 19303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:07:10.441455 19309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05eb3696e5724d4e80acac3e1b99a0b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05eb3696e5724d4e80acac3e1b99a0b8" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 36727 } } }
I20260929 03:07:10.441597 19309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [sys.catalog]: This master's current role is: LEADER
I20260929 03:07:10.441843 19308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05eb3696e5724d4e80acac3e1b99a0b8. Latest consensus state: current_term: 1 leader_uuid: "05eb3696e5724d4e80acac3e1b99a0b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05eb3696e5724d4e80acac3e1b99a0b8" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 36727 } } }
I20260929 03:07:10.441926 19308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8 [sys.catalog]: This master's current role is: LEADER
I20260929 03:07:10.443061 19303 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 03:07:10.449035 19318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:07:10.455909 19318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:07:10.474788 19318 catalog_manager.cc:1347] Generated new cluster ID: d9b768ebc8924d5daf70aaa85292d033
I20260929 03:07:10.475029 19318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:07:10.489265 19318 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:07:10.492347 19318 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:07:10.501155 19318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8: Generated new TSK 0
I20260929 03:07:10.503875 19318 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:07:11.417958 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.209.84:45325/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:36727
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:07:11.596856 19348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:07:11.597183 19348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:07:11.597232 19348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:07:11.601802 19348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:07:11.601886 19348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:07:11.601963 19348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:07:11.609073 19348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.209.84:45325/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:36727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:07:11.610550 19348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:07:11.611941 19348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:07:11.624593 19356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:07:11.624593 19354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:07:11.624708 19353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:07:11.625744 19348 server_base.cc:1034] running on GCE node
I20260929 03:07:11.626291 19348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:07:11.627033 19348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:07:11.628232 19348 hybrid_clock.cc:648] HybridClock initialized: now 1790651231628214 us; error 33 us; skew 500 ppm
Sep 29 03:07:11 dist-test-slave-bdps krb5kdc[18790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:07:11.632802 19348 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:07:11.634447 19348 webserver.cc:460] Webserver started at http://127.7.209.65:35889/ using document root <none> and password file <none>
I20260929 03:07:11.635332 19348 fs_manager.cc:359] Metadata directory not provided
I20260929 03:07:11.635439 19348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:07:11.635753 19348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:07:11 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:07:11.655076 19348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "742d50c07ea045cfbf872daa5ca96651"
format_stamp: "Formatted at 2026-09-29 03:07:11 on dist-test-slave-bdps"
server_key: "e56735cabe0733713c1a45b0288d9c5e"
server_key_iv: "0ba6228a6d9f2656bfdd366774c6ec28"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:11.655905 19348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "742d50c07ea045cfbf872daa5ca96651"
format_stamp: "Formatted at 2026-09-29 03:07:11 on dist-test-slave-bdps"
server_key: "e56735cabe0733713c1a45b0288d9c5e"
server_key_iv: "0ba6228a6d9f2656bfdd366774c6ec28"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:11.660967 19348 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 03:07:11.710280 19365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:11.712859 19348 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.002s sys 0.001s
I20260929 03:07:11.713056 19348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "742d50c07ea045cfbf872daa5ca96651"
format_stamp: "Formatted at 2026-09-29 03:07:11 on dist-test-slave-bdps"
server_key: "e56735cabe0733713c1a45b0288d9c5e"
server_key_iv: "0ba6228a6d9f2656bfdd366774c6ec28"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:11.713277 19348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:07:11.756361 19348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:07:11.760955 19348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:07:11.761193 19348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:07:11.762162 19348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:07:11.763547 19348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:07:11.763648 19348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:11.763726 19348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:07:11.763756 19348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:11.784232 19348 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:33883
I20260929 03:07:11.785688 19348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:07:11.785688 19480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:33883 every 8 connection(s)
I20260929 03:07:11.789510 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 19348
I20260929 03:07:11.789680 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:07:11.790055 8005 external_mini_cluster.cc:1428] Setting key cf4d1fe0942d195b16306f9a02a7b674
Sep 29 03:07:11 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
I20260929 03:07:11.810317 19483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:11.788191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:45627 (local address 127.7.209.126:36727)
0929 03:07:11.790058 (+ 1867us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:11.790063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:11.792323 (+ 2260us) server_negotiation.cc:407] Connection header received
0929 03:07:11.792380 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:11.792383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:11.792447 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:11.792537 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:07:11.794330 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:11.795229 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:11.797135 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:11.797527 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:11.797686 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:11.801660 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:07:11.801665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:07:11.801668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:07:11.801705 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:07:11.803938 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:11.804732 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:11.804735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:11.804736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:11.804864 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:11.805245 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:11.805247 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:11.805248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:11.805579 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:07:11.805717 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:11.809938 (+ 4221us) server_negotiation.cc:299] Negotiation successful
0929 03:07:11.810103 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1744,"thread_start_us":101,"threads_started":1}
I20260929 03:07:11.811877 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:11.788721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:36727 (local address 127.7.209.65:45627)
0929 03:07:11.790818 (+ 2097us) negotiation.cc:107] Waiting for socket to connect
0929 03:07:11.790866 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 03:07:11.791979 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:07:11.792890 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:07:11.792901 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:07:11.793314 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:07:11.794141 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:11.794162 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:11.795407 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:11.795411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:11.796972 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:11.796987 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:11.797905 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:11.797910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:11.798155 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:11.799004 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:07:11.799029 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:07:11.801384 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:07:11.804216 (+ 2832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:07:11.804223 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:07:11.804247 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:07:11.804599 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:07:11.804962 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:07:11.804965 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:07:11.804966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:07:11.805136 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:07:11.809096 (+ 3960us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:07:11.809105 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:07:11.809589 (+ 484us) client_negotiation.cc:769] Sending connection context
0929 03:07:11.809878 (+ 289us) client_negotiation.cc:240] Negotiation successful
0929 03:07:11.810317 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1863,"thread_start_us":173,"threads_started":1}
I20260929 03:07:11.813763 19481 heartbeater.cc:347] Connected to a master server at 127.7.209.126:36727
I20260929 03:07:11.814067 19481 heartbeater.cc:464] Registering TS with master...
I20260929 03:07:11.814767 19481 heartbeater.cc:511] Master 127.7.209.126:36727 requested a full tablet report, sending...
I20260929 03:07:11.816874 19267 ts_manager.cc:194] Registered new tserver with Master: 742d50c07ea045cfbf872daa5ca96651 (127.7.209.65:33883)
I20260929 03:07:11.818732 19267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:45627
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:07:11.836328 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.209.84:45325/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:36727
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:07:11 dist-test-slave-bdps krb5kdc[18790](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790651231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 03:07:12.048956 19487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:07:12.049328 19487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:07:12.049397 19487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:07:12.053818 19487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:07:12.053889 19487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:07:12.053970 19487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:07:12.060317 19487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.209.84:45325/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:36727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:07:12.062180 19487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:07:12.063961 19487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:07:12.085862 19496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:07:12.086516 19493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:07:12.086807 19494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:07:12.087196 19487 server_base.cc:1034] running on GCE node
I20260929 03:07:12.087827 19487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:07:12.088784 19487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:07:12.092056 19487 hybrid_clock.cc:648] HybridClock initialized: now 1790651232091990 us; error 116 us; skew 500 ppm
Sep 29 03:07:12 dist-test-slave-bdps krb5kdc[18790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651232, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:07:12.096814 19487 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:07:12.098269 19487 webserver.cc:460] Webserver started at http://127.7.209.66:45361/ using document root <none> and password file <none>
I20260929 03:07:12.099182 19487 fs_manager.cc:359] Metadata directory not provided
I20260929 03:07:12.099290 19487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:07:12.099608 19487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:07:12 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651232, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:07:12.124790 19487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "7163e84d2f274441a4c5254e663636cb"
format_stamp: "Formatted at 2026-09-29 03:07:12 on dist-test-slave-bdps"
server_key: "1ec313d690430cc06c5fa34365f9596f"
server_key_iv: "1b9bae470e44682c8538b6d0fbf8083e"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:12.125845 19487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "7163e84d2f274441a4c5254e663636cb"
format_stamp: "Formatted at 2026-09-29 03:07:12 on dist-test-slave-bdps"
server_key: "1ec313d690430cc06c5fa34365f9596f"
server_key_iv: "1b9bae470e44682c8538b6d0fbf8083e"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:12.132401 19487 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 03:07:12.160462 19503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:12.163658 19487 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.001s
I20260929 03:07:12.163913 19487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "7163e84d2f274441a4c5254e663636cb"
format_stamp: "Formatted at 2026-09-29 03:07:12 on dist-test-slave-bdps"
server_key: "1ec313d690430cc06c5fa34365f9596f"
server_key_iv: "1b9bae470e44682c8538b6d0fbf8083e"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:12.164148 19487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:07:12.202461 19487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:07:12.207603 19487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:07:12.207861 19487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:07:12.208833 19487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:07:12.210387 19487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:07:12.210460 19487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:12.210520 19487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:07:12.210548 19487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:12.230429 19487 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:39767
I20260929 03:07:12.231865 19487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:07:12.232295 19616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:39767 every 8 connection(s)
I20260929 03:07:12.232743 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 19487
I20260929 03:07:12.232884 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:07:12.233263 8005 external_mini_cluster.cc:1428] Setting key 34e939fcba6926ea467589694fd37345
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
Sep 29 03:07:12 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651232, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:07:12.261855 19483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.235207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:32873 (local address 127.7.209.126:36727)
0929 03:07:12.235359 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:12.235363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:12.239205 (+ 3842us) server_negotiation.cc:407] Connection header received
0929 03:07:12.240872 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:12.240876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:12.240950 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:12.241040 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:07:12.243413 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.244353 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.246346 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.246701 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.246784 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.256930 (+ 10146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:07:12.256935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:07:12.256939 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:07:12.256976 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:07:12.258964 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:12.259680 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:12.259697 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:12.259699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:12.259805 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:12.260247 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:12.260251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:12.260253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:12.260527 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:07:12.260613 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:12.261430 (+ 817us) server_negotiation.cc:299] Negotiation successful
0929 03:07:12.261595 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:07:12.264230 19619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.235847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:36727 (local address 127.7.209.66:32873)
0929 03:07:12.238974 (+ 3127us) negotiation.cc:107] Waiting for socket to connect
0929 03:07:12.239043 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0929 03:07:12.240544 (+ 1501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:07:12.241455 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:07:12.241467 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:07:12.242163 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:07:12.243167 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.243198 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.244583 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.244587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.246189 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.246200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.247031 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.247035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.247278 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.251781 (+ 4503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:07:12.251806 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:07:12.256648 (+ 4842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:07:12.259121 (+ 2473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:07:12.259129 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:07:12.259154 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:07:12.259548 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:07:12.259923 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:07:12.259926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:07:12.259928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:07:12.260131 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:07:12.260694 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:07:12.260702 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:07:12.261103 (+ 401us) client_negotiation.cc:769] Sending connection context
0929 03:07:12.262438 (+ 1335us) client_negotiation.cc:240] Negotiation successful
0929 03:07:12.262884 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2822,"thread_start_us":195,"threads_started":1}
I20260929 03:07:12.266101 19617 heartbeater.cc:347] Connected to a master server at 127.7.209.126:36727
I20260929 03:07:12.266397 19617 heartbeater.cc:464] Registering TS with master...
I20260929 03:07:12.267112 19617 heartbeater.cc:511] Master 127.7.209.126:36727 requested a full tablet report, sending...
I20260929 03:07:12.268571 19267 ts_manager.cc:194] Registered new tserver with Master: 7163e84d2f274441a4c5254e663636cb (127.7.209.66:39767)
I20260929 03:07:12.269419 19267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:32873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:07:12.276564 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.209.84:45325/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:36727
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:07:12.489729 19622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:07:12.490371 19622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:07:12.490523 19622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Sep 29 03:07:12 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
W20260929 03:07:12.497992 19622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:07:12.498070 19622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:07:12.498163 19622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:07:12.506754 19622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.209.84:45325/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:36727
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:07:12.508600 19622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:07:12.510248 19622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:07:12.527374 19622 server_base.cc:1034] running on GCE node
W20260929 03:07:12.527881 19628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:07:12.529125 19630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:07:12.529507 19627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:07:12.530224 19622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:07:12.531350 19622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:07:12.546940 19622 hybrid_clock.cc:648] HybridClock initialized: now 1790651232546868 us; error 98 us; skew 500 ppm
Sep 29 03:07:12 dist-test-slave-bdps krb5kdc[18790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651232, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:07:12.552003 19622 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:07:12.553418 19622 webserver.cc:460] Webserver started at http://127.7.209.67:44617/ using document root <none> and password file <none>
I20260929 03:07:12.554388 19622 fs_manager.cc:359] Metadata directory not provided
I20260929 03:07:12.554500 19622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:07:12.554852 19622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:07:12 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651232, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:07:12.599967 19622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "cde322f8ec674716b0ed38c86729dc1c"
format_stamp: "Formatted at 2026-09-29 03:07:12 on dist-test-slave-bdps"
server_key: "15698255429e806a2f83ff7cdcf65b0d"
server_key_iv: "baf3ef5b16481576cb491cc0b89c0fdd"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:12.601313 19622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "cde322f8ec674716b0ed38c86729dc1c"
format_stamp: "Formatted at 2026-09-29 03:07:12 on dist-test-slave-bdps"
server_key: "15698255429e806a2f83ff7cdcf65b0d"
server_key_iv: "baf3ef5b16481576cb491cc0b89c0fdd"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:12.612653 19622 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.000s
I20260929 03:07:12.638623 19637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:12.641068 19622 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.006s sys 0.000s
I20260929 03:07:12.641299 19622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "cde322f8ec674716b0ed38c86729dc1c"
format_stamp: "Formatted at 2026-09-29 03:07:12 on dist-test-slave-bdps"
server_key: "15698255429e806a2f83ff7cdcf65b0d"
server_key_iv: "baf3ef5b16481576cb491cc0b89c0fdd"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:12.641515 19622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:07:12.659152 19622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:07:12.663388 19622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:07:12.663622 19622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:07:12.664561 19622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:07:12.665915 19622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:07:12.665987 19622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:12.666051 19622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:07:12.666086 19622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:12.692348 19622 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:45373
I20260929 03:07:12.694981 19622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:07:12.697324 19750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:45373 every 8 connection(s)
I20260929 03:07:12.699693 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 19622
I20260929 03:07:12.699836 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:07:12.700184 8005 external_mini_cluster.cc:1428] Setting key 3f43a87f68b4aa4005a9d556f6dc7127
Sep 29 03:07:12 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651232, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:07:12.741429 19483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.712762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:43853 (local address 127.7.209.126:36727)
0929 03:07:12.713880 (+ 1118us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:12.713886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:12.714908 (+ 1022us) server_negotiation.cc:407] Connection header received
0929 03:07:12.718595 (+ 3687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:12.718599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:12.718668 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:12.718736 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:07:12.724684 (+ 5948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.725600 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.731637 (+ 6037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.732011 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.732094 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.735707 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:07:12.735711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:07:12.735714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:07:12.735743 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:07:12.737519 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:12.738203 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:12.738207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:12.738209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:12.738320 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:12.738712 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:12.738715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:12.738717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:12.738953 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:07:12.739047 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:12.741116 (+ 2069us) server_negotiation.cc:299] Negotiation successful
0929 03:07:12.741264 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":951}
I20260929 03:07:12.741434 19753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.709304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:36727 (local address 127.7.209.67:43853)
0929 03:07:12.714602 (+ 5298us) negotiation.cc:107] Waiting for socket to connect
0929 03:07:12.714675 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0929 03:07:12.716636 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:07:12.719290 (+ 2654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:07:12.719303 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:07:12.721242 (+ 1939us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:07:12.724437 (+ 3195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.724473 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.729753 (+ 5280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.729759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.731472 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.731486 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.732374 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.732380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.732621 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.733436 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:07:12.733462 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:07:12.735480 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:07:12.737672 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:07:12.737679 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:07:12.737701 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:07:12.738071 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:07:12.738413 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:07:12.738416 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:07:12.738418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:07:12.738615 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:07:12.739050 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:07:12.739057 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:07:12.739432 (+ 375us) client_negotiation.cc:769] Sending connection context
0929 03:07:12.739650 (+ 218us) client_negotiation.cc:240] Negotiation successful
0929 03:07:12.740044 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4984,"thread_start_us":263,"threads_started":1}
I20260929 03:07:12.743580 19751 heartbeater.cc:347] Connected to a master server at 127.7.209.126:36727
I20260929 03:07:12.743896 19751 heartbeater.cc:464] Registering TS with master...
I20260929 03:07:12.744621 19751 heartbeater.cc:511] Master 127.7.209.126:36727 requested a full tablet report, sending...
I20260929 03:07:12.745980 19267 ts_manager.cc:194] Registered new tserver with Master: cde322f8ec674716b0ed38c86729dc1c (127.7.209.67:45373)
I20260929 03:07:12.746768 19267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:43853
I20260929 03:07:12.753240 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:07:12.766666 19483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.756404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37458 (local address 127.7.209.126:36727)
0929 03:07:12.756542 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:12.756547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:12.756931 (+ 384us) server_negotiation.cc:407] Connection header received
0929 03:07:12.757084 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:12.757087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:12.757152 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:12.757234 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:07:12.758212 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.759042 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.760295 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.760639 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.760743 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.762747 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:07:12.762751 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:07:12.762754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:07:12.762785 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:07:12.764648 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:12.765178 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:12.765182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:12.765184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:12.765288 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:12.765601 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:12.765631 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:12.765633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:12.765885 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:07:12.766019 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:12.766322 (+ 303us) server_negotiation.cc:299] Negotiation successful
0929 03:07:12.766465 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 03:07:12.772341 19267 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:07:12.777025 19267 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:07:12.801450 19766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.792167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:45373 (local address 127.0.0.1:44686)
0929 03:07:12.793764 (+ 1597us) negotiation.cc:107] Waiting for socket to connect
0929 03:07:12.793778 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:07:12.793983 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:07:12.794620 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:07:12.794627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:07:12.794664 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:07:12.794857 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.794864 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.797132 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.797134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.798779 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.798791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.800844 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.800848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.800953 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.800991 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 03:07:12.801114 (+ 123us) client_negotiation.cc:240] Negotiation successful
0929 03:07:12.801262 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1523,"mutex_wait_us":67,"spinlock_wait_cycles":6400,"thread_start_us":135,"threads_started":1}
I20260929 03:07:12.802507 19764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.791164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:33883 (local address 127.0.0.1:45640)
0929 03:07:12.793240 (+ 2076us) negotiation.cc:107] Waiting for socket to connect
0929 03:07:12.793302 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 03:07:12.793567 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:07:12.794620 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:07:12.794627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:07:12.794665 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:07:12.795080 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.795088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.797064 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.797068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.800543 (+ 3475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.800557 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.802028 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.802032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.802160 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.802186 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 03:07:12.802260 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 03:07:12.802345 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1832,"mutex_wait_us":2182,"spinlock_wait_cycles":4864,"thread_start_us":162,"threads_started":1}
I20260929 03:07:12.803215 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.792390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45640 (local address 127.7.209.65:33883)
0929 03:07:12.792716 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:12.792725 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:12.793517 (+ 792us) server_negotiation.cc:407] Connection header received
0929 03:07:12.793860 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:12.793869 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:12.794145 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:12.794428 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:07:12.795794 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.796924 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.800667 (+ 3743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.801680 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.801796 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.801960 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:12.802922 (+ 962us) server_negotiation.cc:299] Negotiation successful
0929 03:07:12.803065 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
I20260929 03:07:12.806917 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.792871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:39767 (local address 127.0.0.1:59660)
0929 03:07:12.794185 (+ 1314us) negotiation.cc:107] Waiting for socket to connect
0929 03:07:12.794197 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:07:12.794388 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:07:12.797512 (+ 3124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:07:12.797515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:07:12.797534 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:07:12.798858 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.798867 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.802785 (+ 3918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.802789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.805096 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.805108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.806481 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.806485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.806590 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.806620 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 03:07:12.806671 (+ 51us) client_negotiation.cc:240] Negotiation successful
0929 03:07:12.806761 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1256,"mutex_wait_us":1156,"thread_start_us":322,"threads_started":1}
I20260929 03:07:12.807533 19765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.791671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44686 (local address 127.7.209.67:45373)
0929 03:07:12.792843 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:12.792874 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:12.793996 (+ 1122us) server_negotiation.cc:407] Connection header received
0929 03:07:12.794120 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:12.794127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:12.794354 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:12.794668 (+ 314us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:07:12.794977 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.797031 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.799019 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.800717 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.800824 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.801013 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:12.807189 (+ 6176us) server_negotiation.cc:299] Negotiation successful
0929 03:07:12.807369 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1050,"thread_start_us":236,"threads_started":1}
I20260929 03:07:12.807758 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.793086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.7.209.66:39767)
0929 03:07:12.795246 (+ 2160us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:12.795254 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:12.795279 (+ 25us) server_negotiation.cc:407] Connection header received
0929 03:07:12.795355 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:12.795360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:12.795534 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:12.799037 (+ 3503us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:07:12.799209 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.800356 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.805255 (+ 4899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.806338 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.807222 (+ 884us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.807400 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:12.807454 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 03:07:12.807555 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1431,"thread_start_us":136,"threads_started":1}
I20260929 03:07:12.808929 19405 tablet_service.cc:1467] Processing CreateTablet for tablet b60e096f3dc2404d9e730b45e7decc2b (DEFAULT_TABLE table=test-table [id=c34cefed415844caa0b6180b79af0b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:07:12.810585 19405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b60e096f3dc2404d9e730b45e7decc2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:07:12.811151 19685 tablet_service.cc:1467] Processing CreateTablet for tablet b60e096f3dc2404d9e730b45e7decc2b (DEFAULT_TABLE table=test-table [id=c34cefed415844caa0b6180b79af0b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:07:12.811182 19541 tablet_service.cc:1467] Processing CreateTablet for tablet b60e096f3dc2404d9e730b45e7decc2b (DEFAULT_TABLE table=test-table [id=c34cefed415844caa0b6180b79af0b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:07:12.812397 19685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b60e096f3dc2404d9e730b45e7decc2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:07:12.812397 19541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b60e096f3dc2404d9e730b45e7decc2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:07:12.821902 19481 heartbeater.cc:503] Master 127.7.209.126:36727 was elected leader, sending a full tablet report...
I20260929 03:07:12.824747 19772 tablet_bootstrap.cc:492] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c: Bootstrap starting.
I20260929 03:07:12.825985 19770 tablet_bootstrap.cc:492] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651: Bootstrap starting.
I20260929 03:07:12.828203 19772 tablet_bootstrap.cc:654] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c: Neither blocks nor log segments found. Creating new log.
I20260929 03:07:12.829321 19772 log.cc:824] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c: Log is configured to *not* fsync() on all Append() calls
I20260929 03:07:12.828881 19770 tablet_bootstrap.cc:654] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651: Neither blocks nor log segments found. Creating new log.
I20260929 03:07:12.830081 19771 tablet_bootstrap.cc:492] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb: Bootstrap starting.
I20260929 03:07:12.831008 19770 log.cc:824] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651: Log is configured to *not* fsync() on all Append() calls
I20260929 03:07:12.833151 19771 tablet_bootstrap.cc:654] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb: Neither blocks nor log segments found. Creating new log.
I20260929 03:07:12.833535 19772 tablet_bootstrap.cc:492] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c: No bootstrap required, opened a new log
I20260929 03:07:12.833775 19772 ts_tablet_manager.cc:1397] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 03:07:12.834345 19771 log.cc:824] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb: Log is configured to *not* fsync() on all Append() calls
I20260929 03:07:12.837247 19770 tablet_bootstrap.cc:492] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651: No bootstrap required, opened a new log
I20260929 03:07:12.837455 19770 ts_tablet_manager.cc:1397] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260929 03:07:12.838169 19772 raft_consensus.cc:348] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "742d50c07ea045cfbf872daa5ca96651" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33883 } } peers { permanent_uuid: "cde322f8ec674716b0ed38c86729dc1c" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 45373 } } peers { permanent_uuid: "7163e84d2f274441a4c5254e663636cb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39767 } }
I20260929 03:07:12.838474 19772 raft_consensus.cc:374] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:07:12.838527 19772 raft_consensus.cc:729] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cde322f8ec674716b0ed38c86729dc1c, State: Initialized, Role: FOLLOWER
I20260929 03:07:12.839056 19772 consensus_queue.cc:260] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "742d50c07ea045cfbf872daa5ca96651" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33883 } } peers { permanent_uuid: "cde322f8ec674716b0ed38c86729dc1c" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 45373 } } peers { permanent_uuid: "7163e84d2f274441a4c5254e663636cb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39767 } }
I20260929 03:07:12.840349 19772 ts_tablet_manager.cc:1428] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260929 03:07:12.841084 19751 heartbeater.cc:503] Master 127.7.209.126:36727 was elected leader, sending a full tablet report...
I20260929 03:07:12.847244 19771 tablet_bootstrap.cc:492] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb: No bootstrap required, opened a new log
I20260929 03:07:12.847591 19771 ts_tablet_manager.cc:1397] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.004s
I20260929 03:07:12.847968 19770 raft_consensus.cc:348] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "742d50c07ea045cfbf872daa5ca96651" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33883 } } peers { permanent_uuid: "cde322f8ec674716b0ed38c86729dc1c" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 45373 } } peers { permanent_uuid: "7163e84d2f274441a4c5254e663636cb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39767 } }
I20260929 03:07:12.848331 19770 raft_consensus.cc:374] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:07:12.848398 19770 raft_consensus.cc:729] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 742d50c07ea045cfbf872daa5ca96651, State: Initialized, Role: FOLLOWER
I20260929 03:07:12.849110 19770 consensus_queue.cc:260] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "742d50c07ea045cfbf872daa5ca96651" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33883 } } peers { permanent_uuid: "cde322f8ec674716b0ed38c86729dc1c" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 45373 } } peers { permanent_uuid: "7163e84d2f274441a4c5254e663636cb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39767 } }
I20260929 03:07:12.850593 19770 ts_tablet_manager.cc:1428] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651: Time spent starting tablet: real 0.013s user 0.002s sys 0.007s
I20260929 03:07:12.852985 19771 raft_consensus.cc:348] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "742d50c07ea045cfbf872daa5ca96651" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33883 } } peers { permanent_uuid: "cde322f8ec674716b0ed38c86729dc1c" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 45373 } } peers { permanent_uuid: "7163e84d2f274441a4c5254e663636cb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39767 } }
I20260929 03:07:12.853376 19771 raft_consensus.cc:374] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:07:12.853468 19771 raft_consensus.cc:729] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7163e84d2f274441a4c5254e663636cb, State: Initialized, Role: FOLLOWER
I20260929 03:07:12.854035 19771 consensus_queue.cc:260] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "742d50c07ea045cfbf872daa5ca96651" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33883 } } peers { permanent_uuid: "cde322f8ec674716b0ed38c86729dc1c" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 45373 } } peers { permanent_uuid: "7163e84d2f274441a4c5254e663636cb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39767 } }
I20260929 03:07:12.855450 19771 ts_tablet_manager.cc:1428] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260929 03:07:12.855888 19617 heartbeater.cc:503] Master 127.7.209.126:36727 was elected leader, sending a full tablet report...
Sep 29 03:07:12 dist-test-slave-bdps krb5kdc[18790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651231, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
W20260929 03:07:12.949926 19752 tablet.cc:2369] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:07:12.983790 19618 tablet.cc:2369] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:07:12.985548 19777 raft_consensus.cc:482] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:07:12.985818 19777 raft_consensus.cc:504] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "742d50c07ea045cfbf872daa5ca96651" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33883 } } peers { permanent_uuid: "cde322f8ec674716b0ed38c86729dc1c" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 45373 } } peers { permanent_uuid: "7163e84d2f274441a4c5254e663636cb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39767 } }
I20260929 03:07:12.987546 19777 leader_election.cc:288] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cde322f8ec674716b0ed38c86729dc1c (127.7.209.67:45373), 7163e84d2f274441a4c5254e663636cb (127.7.209.66:39767)
I20260929 03:07:12.996452 19765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.989383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:34877 (local address 127.7.209.67:45373)
0929 03:07:12.989545 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:12.989551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:12.991295 (+ 1744us) server_negotiation.cc:407] Connection header received
0929 03:07:12.991532 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:12.991535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:12.991596 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:12.991647 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:07:12.992325 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.993218 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.994994 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.995597 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.995683 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.995747 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:12.996211 (+ 464us) server_negotiation.cc:299] Negotiation successful
0929 03:07:12.996301 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:07:12.997181 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.988053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:45373 (local address 127.7.209.65:34877)
0929 03:07:12.991069 (+ 3016us) negotiation.cc:107] Waiting for socket to connect
0929 03:07:12.991086 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:07:12.991394 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:07:12.991829 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:07:12.991832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:07:12.991853 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:07:12.992174 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.992183 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.993455 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.993458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.994871 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.994881 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.995886 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.995889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.996054 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.996080 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 03:07:12.996869 (+ 789us) client_negotiation.cc:240] Negotiation successful
0929 03:07:12.996946 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2933,"thread_start_us":134,"threads_started":1}
I20260929 03:07:12.998010 19769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.989524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:59927 (local address 127.7.209.66:39767)
0929 03:07:12.989683 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:12.989688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:12.993962 (+ 4274us) server_negotiation.cc:407] Connection header received
0929 03:07:12.994154 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:12.994157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:12.994205 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:12.994250 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:07:12.994643 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.995370 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.996696 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.997292 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.997379 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.997427 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:12.997806 (+ 379us) server_negotiation.cc:299] Negotiation successful
0929 03:07:12.997891 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 03:07:12.998148 19703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60e096f3dc2404d9e730b45e7decc2b" candidate_uuid: "742d50c07ea045cfbf872daa5ca96651" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cde322f8ec674716b0ed38c86729dc1c" is_pre_election: true
I20260929 03:07:12.998346 19782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:12.988572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:39767 (local address 127.7.209.65:59927)
0929 03:07:12.993760 (+ 5188us) negotiation.cc:107] Waiting for socket to connect
0929 03:07:12.993774 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:07:12.994037 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:07:12.994376 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:07:12.994379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:07:12.994392 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:07:12.994522 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.994528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.995505 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.995508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.996553 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:12.996563 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:12.997518 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:12.997522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:12.997644 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:12.997676 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 03:07:12.998146 (+ 470us) client_negotiation.cc:240] Negotiation successful
0929 03:07:12.998220 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5123,"thread_start_us":262,"threads_started":1}
I20260929 03:07:12.998617 19703 raft_consensus.cc:2454] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 742d50c07ea045cfbf872daa5ca96651 in term 0.
I20260929 03:07:13.000030 19571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60e096f3dc2404d9e730b45e7decc2b" candidate_uuid: "742d50c07ea045cfbf872daa5ca96651" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7163e84d2f274441a4c5254e663636cb" is_pre_election: true
I20260929 03:07:13.000118 19367 leader_election.cc:302] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 742d50c07ea045cfbf872daa5ca96651, cde322f8ec674716b0ed38c86729dc1c; no voters:
I20260929 03:07:13.000461 19571 raft_consensus.cc:2454] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 742d50c07ea045cfbf872daa5ca96651 in term 0.
I20260929 03:07:13.000517 19777 raft_consensus.cc:2781] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:07:13.000602 19777 raft_consensus.cc:482] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:07:13.000641 19777 raft_consensus.cc:3037] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:07:13.003764 19777 raft_consensus.cc:504] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "742d50c07ea045cfbf872daa5ca96651" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33883 } } peers { permanent_uuid: "cde322f8ec674716b0ed38c86729dc1c" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 45373 } } peers { permanent_uuid: "7163e84d2f274441a4c5254e663636cb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39767 } }
I20260929 03:07:13.004267 19777 leader_election.cc:288] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [CANDIDATE]: Term 1 election: Requested vote from peers cde322f8ec674716b0ed38c86729dc1c (127.7.209.67:45373), 7163e84d2f274441a4c5254e663636cb (127.7.209.66:39767)
I20260929 03:07:13.005002 19571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60e096f3dc2404d9e730b45e7decc2b" candidate_uuid: "742d50c07ea045cfbf872daa5ca96651" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7163e84d2f274441a4c5254e663636cb"
I20260929 03:07:13.004993 19703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b60e096f3dc2404d9e730b45e7decc2b" candidate_uuid: "742d50c07ea045cfbf872daa5ca96651" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cde322f8ec674716b0ed38c86729dc1c"
I20260929 03:07:13.005167 19571 raft_consensus.cc:3037] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:07:13.005198 19703 raft_consensus.cc:3037] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:07:13.007290 19571 raft_consensus.cc:2454] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 742d50c07ea045cfbf872daa5ca96651 in term 1.
I20260929 03:07:13.007305 19703 raft_consensus.cc:2454] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 742d50c07ea045cfbf872daa5ca96651 in term 1.
I20260929 03:07:13.007781 19367 leader_election.cc:302] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 742d50c07ea045cfbf872daa5ca96651, cde322f8ec674716b0ed38c86729dc1c; no voters:
I20260929 03:07:13.008062 19777 raft_consensus.cc:2781] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:07:13.008430 19777 raft_consensus.cc:686] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 1 LEADER]: Becoming Leader. State: Replica: 742d50c07ea045cfbf872daa5ca96651, State: Running, Role: LEADER
I20260929 03:07:13.008888 19777 consensus_queue.cc:237] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "742d50c07ea045cfbf872daa5ca96651" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33883 } } peers { permanent_uuid: "cde322f8ec674716b0ed38c86729dc1c" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 45373 } } peers { permanent_uuid: "7163e84d2f274441a4c5254e663636cb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39767 } }
I20260929 03:07:13.013257 19267 catalog_manager.cc:5495] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 reported cstate change: term changed from 0 to 1, leader changed from <none> to 742d50c07ea045cfbf872daa5ca96651 (127.7.209.65). New cstate: current_term: 1 leader_uuid: "742d50c07ea045cfbf872daa5ca96651" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "742d50c07ea045cfbf872daa5ca96651" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33883 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cde322f8ec674716b0ed38c86729dc1c" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 45373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7163e84d2f274441a4c5254e663636cb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39767 } health_report { overall_health: UNKNOWN } } }
W20260929 03:07:13.038065 19482 tablet.cc:2369] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:07:13.047143 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:13.040931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45646 (local address 127.7.209.65:33883)
0929 03:07:13.041130 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:13.041135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:13.041153 (+ 18us) server_negotiation.cc:407] Connection header received
0929 03:07:13.041258 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:13.041261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:13.041344 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:13.041463 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:07:13.042004 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:13.042913 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:13.044167 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:13.044528 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:13.044643 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:13.045171 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:07:13.045995 (+ 824us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:07:13.046139 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:13.046797 (+ 658us) server_negotiation.cc:299] Negotiation successful
0929 03:07:13.046879 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260929 03:07:13.059408 19571 raft_consensus.cc:1261] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb [term 1 FOLLOWER]: Refusing update from remote peer 742d50c07ea045cfbf872daa5ca96651: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:07:13.060444 19777 consensus_queue.cc:1035] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7163e84d2f274441a4c5254e663636cb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:07:13.071655 19703 raft_consensus.cc:1261] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c [term 1 FOLLOWER]: Refusing update from remote peer 742d50c07ea045cfbf872daa5ca96651: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:07:13.072693 19777 consensus_queue.cc:1035] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cde322f8ec674716b0ed38c86729dc1c" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 45373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:07:13.089844 19786 mvcc.cc:205] Tried to move back new op lower bound from 7334507450596691968 to 7334507450408591360. Current Snapshot: MvccSnapshot[applied={T|T < 7334507450596691968}]
I20260929 03:07:13.090512 19787 mvcc.cc:205] Tried to move back new op lower bound from 7334507450596691968 to 7334507450408591360. Current Snapshot: MvccSnapshot[applied={T|T < 7334507450596691968}]
I20260929 03:07:13.090957 19789 mvcc.cc:205] Tried to move back new op lower bound from 7334507450596691968 to 7334507450408591360. Current Snapshot: MvccSnapshot[applied={T|T < 7334507450596691968}]
I20260929 03:07:13.104444 19267 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:07:13.104681 19267 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:07:13.108014 19267 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05eb3696e5724d4e80acac3e1b99a0b8: Sending DeleteTablet for 3 replicas of tablet b60e096f3dc2404d9e730b45e7decc2b
I20260929 03:07:13.109679 19685 tablet_service.cc:1514] Processing DeleteTablet for tablet b60e096f3dc2404d9e730b45e7decc2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:07:13 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:44686
I20260929 03:07:13.110703 19541 tablet_service.cc:1514] Processing DeleteTablet for tablet b60e096f3dc2404d9e730b45e7decc2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:07:13 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:59660
I20260929 03:07:13.111845 19405 tablet_service.cc:1514] Processing DeleteTablet for tablet b60e096f3dc2404d9e730b45e7decc2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:07:13 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:45640
I20260929 03:07:13.112512 19798 tablet_replica.cc:330] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651: stopping tablet replica
I20260929 03:07:13.112969 19798 raft_consensus.cc:2229] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:07:13.113246 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 19348
I20260929 03:07:13.113564 19798 raft_consensus.cc:2258] T b60e096f3dc2404d9e730b45e7decc2b P 742d50c07ea045cfbf872daa5ca96651 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:07:13.121989 19796 tablet_replica.cc:330] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c: stopping tablet replica
I20260929 03:07:13.122582 19796 raft_consensus.cc:2229] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:07:13.123154 19796 raft_consensus.cc:2258] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:07:13.123720 19797 tablet_replica.cc:330] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb: stopping tablet replica
I20260929 03:07:13.124116 19797 raft_consensus.cc:2229] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb [term 1 FOLLOWER]: Raft consensus shutting down.
W20260929 03:07:13.124223 19253 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:33883 (error 108)
I20260929 03:07:13.124505 19797 raft_consensus.cc:2258] T b60e096f3dc2404d9e730b45e7decc2b P 7163e84d2f274441a4c5254e663636cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:07:13.125540 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 19487
I20260929 03:07:13.127003 19796 ts_tablet_manager.cc:1905] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:07:13.130472 19796 ts_tablet_manager.cc:1918] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:07:13.130586 19796 log.cc:1196] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/b60e096f3dc2404d9e730b45e7decc2b
I20260929 03:07:13.130963 19796 ts_tablet_manager.cc:1939] T b60e096f3dc2404d9e730b45e7decc2b P cde322f8ec674716b0ed38c86729dc1c: Deleting consensus metadata
I20260929 03:07:13.132612 19252 catalog_manager.cc:4841] TS cde322f8ec674716b0ed38c86729dc1c (127.7.209.67:45373): tablet b60e096f3dc2404d9e730b45e7decc2b (table test-table [id=c34cefed415844caa0b6180b79af0b36]) successfully deleted
I20260929 03:07:13.133572 19768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:13.133196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:33883 (local address 127.0.0.1:45654)
0929 03:07:13.133362 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 03:07:13.133459 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:33883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 03:07:13.133880 19253 catalog_manager.cc:4584] TS 742d50c07ea045cfbf872daa5ca96651 (127.7.209.65:33883): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b60e096f3dc2404d9e730b45e7decc2b: Network error: Client connection negotiation failed: client connection to 127.7.209.65:33883: connect: Connection refused (error 111)
W20260929 03:07:13.137692 19768 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:07:13.136027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:39767 (local address 127.0.0.1:59666)
0929 03:07:13.136219 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0929 03:07:13.136236 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:07:13.136337 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:07:13.137553 (+ 1216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.66:39767: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":100}
I20260929 03:07:13.139333 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 19622
I20260929 03:07:13.148206 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 19235
I20260929 03:07:13.155900 8005 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 03:07:13.490715 8005 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 03:07:13.490881 8005 mini_ranger.cc:66] Stopping Ranger...
I20260929 03:07:13.826468 8005 mini_ranger.cc:68] Stopped Ranger
2026-09-29 03:07:13.826 UTC [18820] LOG: received smart shutdown request
2026-09-29 03:07:13.830 UTC [18820] LOG: background worker "logical replication launcher" (PID 18829) exited with exit code 1
2026-09-29 03:07:13.830 UTC [18824] LOG: shutting down
2026-09-29 03:07:13.988 UTC [18820] LOG: database system is shut down
2026-09-29T03:07:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50038 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:07:14 dist-test-slave-bdps krb5kdc[19808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:07:14 dist-test-slave-bdps krb5kdc[19808](info): set up 2 sockets
Sep 29 03:07:14 dist-test-slave-bdps krb5kdc[19808](info): commencing operation
krb5kdc: starting...
W20260929 03:07:15.054064 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:07:15 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:07:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:07:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.209.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.209.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/rangeradmin_127.7.209.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/rangeradmin_127.7.209.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.209.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.209.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/rangerlookup_127.7.209.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/rangerlookup_127.7.209.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.209.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.209.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/HTTP_127.7.209.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/HTTP_127.7.209.84@KRBTEST.COM.keytab.
I20260929 03:07:15.228415 8005 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskC1pyL7/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/postgres -l logfile start
2026-09-29 03:07:17.495 UTC [19838] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 03:07:17.495 UTC [19838] LOG: listening on IPv4 address "127.7.209.84", port 32769
2026-09-29 03:07:17.498 UTC [19838] LOG: listening on Unix socket "/tmp/.s.PGSQL.32769"
2026-09-29 03:07:17.519 UTC [19841] LOG: database system was shut down at 2026-09-29 03:07:16 UTC
2026-09-29 03:07:17.523 UTC [19838] LOG: database system is ready to accept connections
I20260929 03:07:18.401793 8005 mini_postgres.cc:96] Postgres bound to 32769
2026-09-29 03:07:18.410 UTC [19849] FATAL: database "slave" does not exist
127.7.209.84:32769 - accepting connections
I20260929 03:07:18.410878 8005 mini_ranger.cc:161] Starting Ranger...
I20260929 03:07:18.427441 8005 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 03:07:18.839319 8005 mini_ranger.cc:87] Created ranger Postgres database
I20260929 03:07:18.839412 8005 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-admin
2026-09-29 03:07:19,105 [I] DB FLAVOR :POSTGRES
2026-09-29 03:07:19,105 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 03:07:19,105 [I] Checking connection..
2026-09-29 03:07:19,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 03:07:19,378 [I] Checking connection passed.
2026-09-29 03:07:19,379 [I] --------- Verifying version history table ---------
2026-09-29 03:07:19,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 03:07:19,651 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 03:07:19,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 03:07:19,909 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 03:07:19,909 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 03:07:19,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 03:07:20.167 UTC [19913] WARNING: there is no transaction in progress
2026-09-29 03:07:20,174 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 03:07:20,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 03:07:20,412 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 03:07:20,412 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 03:07:20,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 03:07:20,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 03:07:20,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-bdps.c.gcp-upstream.internal','N') ;"
2026-09-29 03:07:21,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 03:07:21,412 [I] Table x_portal_user does not exist in database ranger
2026-09-29 03:07:21,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 03:07:21,685 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 03:07:21,685 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 03:07:21,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 03:07:22,197 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 03:07:22,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 03:07:23.269 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.284 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.301 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.316 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.332 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.348 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.403 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.410 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.424 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.435 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.445 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.461 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.470 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.476 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.482 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:23.490 UTC [20021] WARNING: there is no transaction in progress
2026-09-29 03:07:24,078 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 03:07:24,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 03:07:24,339 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 03:07:24,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 03:07:24,613 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 03:07:24,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 03:07:24,872 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 03:07:24,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 03:07:25,136 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 03:07:25,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-bdps.c.gcp-upstream.internal';"
2026-09-29 03:07:25,626 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 03:07:25,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 03:07:25,896 [I] DB_PATCHES have already been applied
I20260929 03:07:25.902725 8005 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-admin:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskC1pyL7/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 03:07:25.902808 8005 mini_ranger.cc:193] Using host: 127.7.209.84
I20260929 03:07:25.907522 8005 mini_ranger.cc:227] Ranger admin URL: http://127.7.209.84:51551
Sep 29, 2026 3:07:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 3:07:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 3:07:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 3:07:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 3:07:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 3:07:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 3:07:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 03:07:26.912160 8005 mini_ranger.cc:160] Time spent starting Ranger: real 8.501s user 0.003s sys 0.010s
Sep 29, 2026 3:07:48 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 3:07:48 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 3:07:48 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 3:07:48 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 3:07:48 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 3:07:49 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 03:07:49.810096 8005 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.209.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.209.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/rangerkms_127.7.209.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.209.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/rangerkms_127.7.209.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.209.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.209.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.209.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/HTTP_127.7.209.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.209.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/HTTP_127.7.209.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 03:07:49 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651269, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 03:07:49.918323 8005 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 03:07:49.930220 8005 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 03:07:50.699872 8005 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 03:07:50.699995 8005 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-kms
I20260929 03:07:50.700017 8005 mini_ranger_kms.cc:227] Using postgres at 127.7.209.84:32769
2026-09-29 03:07:50,943 [I] DB FLAVOR :POSTGRES
2026-09-29 03:07:50,944 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 03:07:50,944 [I] Checking connection
2026-09-29 03:07:50,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 03:07:51,201 [I] connection success
2026-09-29 03:07:51,201 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 03:07:51,201 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 03:07:51,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 03:07:51,471 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 03:07:51,472 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 03:07:51,472 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 03:07:51,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.209.84:32769/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 03:07:51,737 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 03:07:52.278431 8005 mini_ranger_kms.cc:310] Created kms service
I20260929 03:07:52.479147 8005 mini_ranger_kms.cc:326] Created kudu user
I20260929 03:07:52.528365 8005 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 03:07:52 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651269, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:07:52.957377 8005 mini_ranger_kms.cc:384] Added ranger policy
I20260929 03:07:52.957600 8005 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-kms:/tmp/dist-test-taskC1pyL7/build/debug/bin/postgresql.jar:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskC1pyL7/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 03:07:52.957659 8005 mini_ranger_kms.cc:240] Using host: 127.7.209.84
I20260929 03:07:52.962849 8005 mini_ranger_kms.cc:276] Ranger KMS PID: 20202
I20260929 03:07:52.962950 8005 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.209.84:43621
Sep 29, 2026 3:07:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 3:07:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 3:07:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 3:07:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 3:07:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 3:07:54 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 03:07:55.008558 8005 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.090s user 0.026s sys 0.203s
I20260929 03:07:55.008728 8005 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 03:07:55.008823 8005 mini_ranger_kms.cc:401] 127.7.209.84:43621/kms/v1/keys
Sep 29 03:07:55 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790651275, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.209.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 03:07:55 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651275, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.209.84@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
Sep 29 03:07:56 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651275, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.209.84@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
Sep 29 03:07:58 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:07:58.447005 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:42605
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:42605
--ranger_config_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.209.84:43621/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:07:58.575161 20252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:07:58.575462 20252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:07:58.575500 20252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:07:58.579563 20252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:07:58.579634 20252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:07:58.579653 20252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:07:58.579667 20252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:07:58.579699 20252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:07:58.585640 20252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.209.84:43621/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:42605
--ranger_config_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:42605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:07:58.587190 20252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:07:58.588606 20252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:07:58.599535 20252 server_base.cc:1034] running on GCE node
W20260929 03:07:58.599558 20260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:07:58.601217 20258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:07:58.601290 20257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:07:58.602020 20252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:07:58.603456 20252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:07:58.604624 20252 hybrid_clock.cc:648] HybridClock initialized: now 1790651278604607 us; error 33 us; skew 500 ppm
Sep 29 03:07:58 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651278, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:07:58.610853 20252 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:07:58.611968 20252 webserver.cc:460] Webserver started at http://127.7.209.126:40221/ using document root <none> and password file <none>
I20260929 03:07:58.612607 20252 fs_manager.cc:359] Metadata directory not provided
I20260929 03:07:58.612663 20252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:07:58.612913 20252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:07:58 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651278, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:07:58.683624 20252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe"
format_stamp: "Formatted at 2026-09-29 03:07:58 on dist-test-slave-bdps"
server_key: "b3fd1e007adc78606cccba02ab55666c"
server_key_iv: "34b4a57a769e9ca097c6ed800942e3cc"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:58.684386 20252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe"
format_stamp: "Formatted at 2026-09-29 03:07:58 on dist-test-slave-bdps"
server_key: "b3fd1e007adc78606cccba02ab55666c"
server_key_iv: "34b4a57a769e9ca097c6ed800942e3cc"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:58.689270 20252 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 03:07:58.710577 20267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:58.712462 20252 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 03:07:58.712621 20252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe"
format_stamp: "Formatted at 2026-09-29 03:07:58 on dist-test-slave-bdps"
server_key: "b3fd1e007adc78606cccba02ab55666c"
server_key_iv: "34b4a57a769e9ca097c6ed800942e3cc"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:58.712805 20252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:07:58.737797 20252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:07:58.741161 20252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:07:58.741379 20252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:07:58.751668 20252 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:42605
I20260929 03:07:58.751715 20319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:42605 every 8 connection(s)
I20260929 03:07:58.752835 20252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:07:58.757129 20320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:07:58.757768 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 20252
I20260929 03:07:58.757892 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:07:58.758212 8005 external_mini_cluster.cc:1428] Setting key 99d7342a50f6524a46e69028817f4c46
I20260929 03:07:58.765427 20320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe: Bootstrap starting.
Sep 29 03:07:58 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:07:58.768452 20320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe: Neither blocks nor log segments found. Creating new log.
I20260929 03:07:58.769425 20320 log.cc:824] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe: Log is configured to *not* fsync() on all Append() calls
I20260929 03:07:58.771812 20320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe: No bootstrap required, opened a new log
I20260929 03:07:58.774580 20323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:58.759805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56062 (local address 127.7.209.126:42605)
0929 03:07:58.760251 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:58.760274 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:58.760323 (+ 49us) server_negotiation.cc:407] Connection header received
0929 03:07:58.761191 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:58.761220 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:58.761741 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:58.762171 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:07:58.762900 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:58.764476 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:58.765355 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:58.766077 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:58.766233 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:58.768977 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:07:58.768983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:07:58.769000 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:07:58.769031 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:07:58.771501 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:58.771905 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:58.771911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:58.771917 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:58.772052 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:58.772301 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:58.772304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:58.772305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:58.772838 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:07:58.773052 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:58.773215 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 03:07:58.773509 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":108,"threads_started":1}
I20260929 03:07:58.775734 20320 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42605 } }
I20260929 03:07:58.775971 20320 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:07:58.776021 20320 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf2ab41766484e4c99ee5d5d7e7cd2fe, State: Initialized, Role: FOLLOWER
I20260929 03:07:58.776499 20320 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42605 } }
I20260929 03:07:58.776628 20320 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:07:58.776696 20320 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:07:58.776798 20320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:07:58.777798 20320 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42605 } }
I20260929 03:07:58.778160 20320 leader_election.cc:302] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf2ab41766484e4c99ee5d5d7e7cd2fe; no voters:
I20260929 03:07:58.778462 20320 leader_election.cc:288] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:07:58.778544 20325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:07:58.778801 20325 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 1 LEADER]: Becoming Leader. State: Replica: bf2ab41766484e4c99ee5d5d7e7cd2fe, State: Running, Role: LEADER
I20260929 03:07:58.779182 20325 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42605 } }
I20260929 03:07:58.779814 20320 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:07:58.780884 20326 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42605 } } }
I20260929 03:07:58.781013 20326 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [sys.catalog]: This master's current role is: LEADER
I20260929 03:07:58.781276 20327 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf2ab41766484e4c99ee5d5d7e7cd2fe. Latest consensus state: current_term: 1 leader_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42605 } } }
I20260929 03:07:58.781363 20327 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [sys.catalog]: This master's current role is: LEADER
I20260929 03:07:58.781468 20334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:07:58.785524 20334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:07:58.786060 20320 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 03:07:58.797106 20334 catalog_manager.cc:1347] Generated new cluster ID: 25b9c7e121164fd5a236d68362b6777b
I20260929 03:07:58.797235 20334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:07:58.808314 20334 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:07:58.809602 20334 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:07:58.822013 20334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe: Generated new TSK 0
I20260929 03:07:58.822975 20334 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:07:59.482844 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.209.84:43621/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:42605
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:07:59.616720 20365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:07:59.617092 20365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:07:59.617156 20365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:07:59.621683 20365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:07:59.621749 20365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:07:59.621819 20365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:07:59.627674 20365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.209.84:43621/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:42605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:07:59.629145 20365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:07:59.630563 20365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:07:59.642748 20371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:07:59.642915 20365 server_base.cc:1034] running on GCE node
W20260929 03:07:59.642758 20370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:07:59.642771 20373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:07:59.643739 20365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:07:59.644586 20365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:07:59.645905 20365 hybrid_clock.cc:648] HybridClock initialized: now 1790651279645752 us; error 171 us; skew 500 ppm
Sep 29 03:07:59 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651279, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:07:59.649828 20365 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:07:59.651051 20365 webserver.cc:460] Webserver started at http://127.7.209.65:38163/ using document root <none> and password file <none>
I20260929 03:07:59.651858 20365 fs_manager.cc:359] Metadata directory not provided
I20260929 03:07:59.651954 20365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:07:59.652302 20365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:07:59 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651279, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:07:59.670091 20365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba"
format_stamp: "Formatted at 2026-09-29 03:07:59 on dist-test-slave-bdps"
server_key: "9b4dbb8aae096e3c4177dd8eaf18821a"
server_key_iv: "c6690618dd7f6469451d58b71078f407"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:59.671034 20365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba"
format_stamp: "Formatted at 2026-09-29 03:07:59 on dist-test-slave-bdps"
server_key: "9b4dbb8aae096e3c4177dd8eaf18821a"
server_key_iv: "c6690618dd7f6469451d58b71078f407"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:59.676222 20365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 03:07:59.693776 20382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:59.695989 20365 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:07:59.696228 20365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba"
format_stamp: "Formatted at 2026-09-29 03:07:59 on dist-test-slave-bdps"
server_key: "9b4dbb8aae096e3c4177dd8eaf18821a"
server_key_iv: "c6690618dd7f6469451d58b71078f407"
server_key_version: "kuduclusterkey@0"
I20260929 03:07:59.696496 20365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:07:59.723857 20365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:07:59.727216 20365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:07:59.727415 20365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:07:59.728157 20365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:07:59.729175 20365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:07:59.729236 20365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:07:59.729295 20365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:07:59.729336 20365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260929 03:07:59.740756 20365 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:44543
I20260929 03:07:59.740810 20495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:44543 every 8 connection(s)
I20260929 03:07:59.741909 20365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:07:59.742471 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 20365
I20260929 03:07:59.742612 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:07:59.742995 8005 external_mini_cluster.cc:1428] Setting key b16791a0842344166b5df7a48532a830
Sep 29 03:07:59 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651279, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:07:59.766525 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:59.750466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:50179 (local address 127.7.209.126:42605)
0929 03:07:59.751168 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0929 03:07:59.751179 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:07:59.751799 (+ 620us) server_negotiation.cc:407] Connection header received
0929 03:07:59.753226 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:07:59.753230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:07:59.753295 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:07:59.753369 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:07:59.755404 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:59.756360 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:59.758206 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:07:59.758578 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:07:59.758658 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:59.762089 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:07:59.762093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:07:59.762096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:07:59.762123 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:07:59.764030 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:59.764610 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:59.764614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:59.764617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:59.764724 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:07:59.765116 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:07:59.765120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:07:59.765122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:07:59.765350 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:07:59.765429 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 03:07:59.766194 (+ 765us) server_negotiation.cc:299] Negotiation successful
0929 03:07:59.766334 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":163,"threads_started":1}
I20260929 03:07:59.768652 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:07:59.750953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42605 (local address 127.7.209.65:50179)
0929 03:07:59.751606 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0929 03:07:59.751675 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0929 03:07:59.752956 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:07:59.754002 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:07:59.754012 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:07:59.754435 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:07:59.755220 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:59.755243 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:59.756524 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:59.756527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:59.758063 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:07:59.758074 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:07:59.758906 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:07:59.758910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:07:59.759116 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:07:59.759911 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:07:59.759936 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:07:59.761892 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:07:59.764166 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:07:59.764172 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:07:59.764189 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:07:59.764503 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:07:59.764825 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:07:59.764828 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:07:59.764830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:07:59.765011 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:07:59.765494 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:07:59.765501 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:07:59.765895 (+ 394us) client_negotiation.cc:769] Sending connection context
0929 03:07:59.767017 (+ 1122us) client_negotiation.cc:240] Negotiation successful
0929 03:07:59.767393 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":182,"threads_started":1}
I20260929 03:07:59.770553 20496 heartbeater.cc:347] Connected to a master server at 127.7.209.126:42605
I20260929 03:07:59.770845 20496 heartbeater.cc:464] Registering TS with master...
I20260929 03:07:59.771490 20496 heartbeater.cc:511] Master 127.7.209.126:42605 requested a full tablet report, sending...
I20260929 03:07:59.773522 20284 ts_manager.cc:194] Registered new tserver with Master: a1c5e4cfb1eb4c96b1060ddd97a123ba (127.7.209.65:44543)
I20260929 03:07:59.774873 20284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:50179
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:07:59.814496 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.209.84:43621/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:42605
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:07:59 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790651279, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 03:07:59.974571 20504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:07:59.974903 20504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:07:59.974968 20504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:07:59.979092 20504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:07:59.979166 20504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:07:59.979254 20504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:07:59.985556 20504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.209.84:43621/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:42605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:07:59.987324 20504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:07:59.988852 20504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:00.002619 20513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:00.002483 20511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:00.002643 20510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:00.002815 20504 server_base.cc:1034] running on GCE node
I20260929 03:08:00.003504 20504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:00.004271 20504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:00.005448 20504 hybrid_clock.cc:648] HybridClock initialized: now 1790651280005427 us; error 39 us; skew 500 ppm
Sep 29 03:08:00 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651280, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:00.009846 20504 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:08:00.010991 20504 webserver.cc:460] Webserver started at http://127.7.209.66:38159/ using document root <none> and password file <none>
I20260929 03:08:00.011651 20504 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:00.011725 20504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:00.012248 20504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:08:00 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651280, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:08:00.030865 20504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "8b819b06846945ec8cff5e23bdc54ced"
format_stamp: "Formatted at 2026-09-29 03:08:00 on dist-test-slave-bdps"
server_key: "27b34787ce8962aa275e02336ade76e2"
server_key_iv: "76eb901124fbc9cf97950e1dc40b3dd4"
server_key_version: "kuduclusterkey@0"
I20260929 03:08:00.031896 20504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "8b819b06846945ec8cff5e23bdc54ced"
format_stamp: "Formatted at 2026-09-29 03:08:00 on dist-test-slave-bdps"
server_key: "27b34787ce8962aa275e02336ade76e2"
server_key_iv: "76eb901124fbc9cf97950e1dc40b3dd4"
server_key_version: "kuduclusterkey@0"
I20260929 03:08:00.038645 20504 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260929 03:08:00.066087 20520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:00.068487 20504 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260929 03:08:00.068676 20504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "8b819b06846945ec8cff5e23bdc54ced"
format_stamp: "Formatted at 2026-09-29 03:08:00 on dist-test-slave-bdps"
server_key: "27b34787ce8962aa275e02336ade76e2"
server_key_iv: "76eb901124fbc9cf97950e1dc40b3dd4"
server_key_version: "kuduclusterkey@0"
I20260929 03:08:00.068886 20504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:00.123400 20504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:00.127924 20504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:00.128232 20504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:00.129104 20504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:00.130429 20504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:00.130544 20504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:00.130648 20504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:00.130736 20504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:00.153432 20504 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:44301
I20260929 03:08:00.153782 20633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:44301 every 8 connection(s)
I20260929 03:08:00.155251 20504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:08:00.160436 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 20504
I20260929 03:08:00.160604 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:08:00.160986 8005 external_mini_cluster.cc:1428] Setting key 0d996dade4a348800d74281940f45cc8
Sep 29 03:08:00 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651280, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:00.175194 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:00.159730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:52853 (local address 127.7.209.126:42605)
0929 03:08:00.159931 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:00.159936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:00.162167 (+ 2231us) server_negotiation.cc:407] Connection header received
0929 03:08:00.163368 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:00.163371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:00.163431 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:00.163509 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:00.164919 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:00.165505 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:00.167350 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:00.167717 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:00.167839 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:00.170992 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:00.170995 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:00.170999 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:00.171031 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:00.172779 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:00.173332 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:00.173336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:00.173338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:00.173442 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:00.173797 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:00.173801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:00.173803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:00.174061 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:00.174170 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:00.174799 (+ 629us) server_negotiation.cc:299] Negotiation successful
0929 03:08:00.174954 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 03:08:00.176517 20636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:00.160493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42605 (local address 127.7.209.66:52853)
0929 03:08:00.161716 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:00.162042 (+ 326us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:00.163112 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:00.163764 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:00.163774 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:00.164113 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:00.164764 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:00.164781 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:00.165662 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:00.165666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:00.167196 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:00.167207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:00.168030 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:00.168033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:00.168254 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:00.168934 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:00.168955 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:00.170792 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:00.172908 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:00.172914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:00.172929 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:00.173218 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:00.173520 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:00.173522 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:00.173523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:00.173699 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:00.174206 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:00.174212 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:00.174512 (+ 300us) client_negotiation.cc:769] Sending connection context
0929 03:08:00.174707 (+ 195us) client_negotiation.cc:240] Negotiation successful
0929 03:08:00.175050 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":214,"threads_started":1}
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:08:00.178109 20634 heartbeater.cc:347] Connected to a master server at 127.7.209.126:42605
I20260929 03:08:00.178440 20634 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:00.179157 20634 heartbeater.cc:511] Master 127.7.209.126:42605 requested a full tablet report, sending...
I20260929 03:08:00.180622 20284 ts_manager.cc:194] Registered new tserver with Master: 8b819b06846945ec8cff5e23bdc54ced (127.7.209.66:44301)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
I20260929 03:08:00.181427 20284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:52853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:08:00.199098 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.209.84:43621/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:42605
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:00.395927 20639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:00.396557 20639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:00.396770 20639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:00.403283 20639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:00.403462 20639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:00.403632 20639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
Sep 29 03:08:00 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651279, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:08:00.412151 20639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.209.84:43621/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:42605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:00.414285 20639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:00.416040 20639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:00.433007 20647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:00.434872 20639 server_base.cc:1034] running on GCE node
W20260929 03:08:00.435781 20645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:00.436035 20644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:00.436699 20639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:00.437700 20639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:00.440984 20639 hybrid_clock.cc:648] HybridClock initialized: now 1790651280440772 us; error 220 us; skew 500 ppm
Sep 29 03:08:00 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651280, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:00.445996 20639 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:08:00.447404 20639 webserver.cc:460] Webserver started at http://127.7.209.67:36913/ using document root <none> and password file <none>
I20260929 03:08:00.448207 20639 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:00.448293 20639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:00.448575 20639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 03:08:00 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651280, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:08:00.485529 20639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "7fe67f2a265e4f38939653ac9b021453"
format_stamp: "Formatted at 2026-09-29 03:08:00 on dist-test-slave-bdps"
server_key: "75e052d3d4a686f7fcaeb0ec3f960ad8"
server_key_iv: "dbfaa929dc3be39c9198d5b5aebf860c"
server_key_version: "kuduclusterkey@0"
I20260929 03:08:00.486632 20639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "7fe67f2a265e4f38939653ac9b021453"
format_stamp: "Formatted at 2026-09-29 03:08:00 on dist-test-slave-bdps"
server_key: "75e052d3d4a686f7fcaeb0ec3f960ad8"
server_key_iv: "dbfaa929dc3be39c9198d5b5aebf860c"
server_key_version: "kuduclusterkey@0"
I20260929 03:08:00.499713 20639 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.004s sys 0.000s
I20260929 03:08:00.565645 20654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:00.568140 20639 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.004s sys 0.002s
I20260929 03:08:00.568324 20639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "7fe67f2a265e4f38939653ac9b021453"
format_stamp: "Formatted at 2026-09-29 03:08:00 on dist-test-slave-bdps"
server_key: "75e052d3d4a686f7fcaeb0ec3f960ad8"
server_key_iv: "dbfaa929dc3be39c9198d5b5aebf860c"
server_key_version: "kuduclusterkey@0"
I20260929 03:08:00.568500 20639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:00.612557 20639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:00.617100 20639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:00.617340 20639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:00.618448 20639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:00.619824 20639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:00.619894 20639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:00.620009 20639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:00.620070 20639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:00.643644 20639 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:34945
I20260929 03:08:00.643925 20767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:34945 every 8 connection(s)
I20260929 03:08:00.645105 20639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:08:00.645661 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 20639
I20260929 03:08:00.645814 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:08:00.646235 8005 external_mini_cluster.cc:1428] Setting key 5fca78f9fe8cacddd6849ac615bc20f2
Sep 29 03:08:00 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651280, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:00.666271 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:00.649329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:44081 (local address 127.7.209.126:42605)
0929 03:08:00.649498 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:00.649503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:00.650794 (+ 1291us) server_negotiation.cc:407] Connection header received
0929 03:08:00.652479 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:00.652483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:00.652544 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:00.652631 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:00.654536 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:00.655412 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:00.657407 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:00.657799 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:00.657917 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:00.661655 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:00.661659 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:00.661662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:00.661696 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:00.663455 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:00.664142 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:00.664146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:00.664148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:00.664272 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:00.664711 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:00.664715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:00.664717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:00.664999 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:00.665114 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:00.665894 (+ 780us) server_negotiation.cc:299] Negotiation successful
0929 03:08:00.666051 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 03:08:00.668028 20770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:00.649991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42605 (local address 127.7.209.67:44081)
0929 03:08:00.650614 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:00.650679 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:00.652220 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:00.652848 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:00.652858 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:00.653385 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:00.654337 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:00.654362 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:00.655588 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:00.655592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:00.657252 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:00.657264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:00.658024 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:00.658029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:00.658289 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:00.659163 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:00.659186 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:00.661434 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:00.663584 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:00.663591 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:00.663613 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:00.664024 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:00.664375 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:00.664378 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:00.664380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:00.664600 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:00.665116 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:00.665124 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:00.665539 (+ 415us) client_negotiation.cc:769] Sending connection context
0929 03:08:00.665834 (+ 295us) client_negotiation.cc:240] Negotiation successful
0929 03:08:00.666285 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":113,"threads_started":1}
I20260929 03:08:00.669821 20768 heartbeater.cc:347] Connected to a master server at 127.7.209.126:42605
I20260929 03:08:00.670212 20768 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:00.670994 20768 heartbeater.cc:511] Master 127.7.209.126:42605 requested a full tablet report, sending...
I20260929 03:08:00.672672 20282 ts_manager.cc:194] Registered new tserver with Master: 7fe67f2a265e4f38939653ac9b021453 (127.7.209.67:34945)
I20260929 03:08:00.673470 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:08:00.673516 20282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:44081
I20260929 03:08:00.674757 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 20365
I20260929 03:08:00.683900 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 20504
I20260929 03:08:00.692847 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 20639
I20260929 03:08:00.699046 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 20252
I20260929 03:08:00.706965 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:42605
--webserver_interface=127.7.209.126
--webserver_port=40221
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:42605
--ranger_config_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.209.84:43621/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.209.84:43621/kms with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:00.831516 20771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:00.831841 20771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:00.831883 20771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:00.836009 20771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:08:00.836082 20771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:00.836100 20771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:00.836114 20771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:08:00.836128 20771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:08:00.841876 20771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.209.84:43621/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:42605
--ranger_config_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:42605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=40221
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:00.843122 20771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:00.844345 20771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:00.853480 20779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:00.853540 20771 server_base.cc:1034] running on GCE node
W20260929 03:08:00.853466 20776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:00.853467 20777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:00.854346 20771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:00.855581 20771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:00.856755 20771 hybrid_clock.cc:648] HybridClock initialized: now 1790651280856738 us; error 29 us; skew 500 ppm
Sep 29 03:08:00 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651280, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:00.859712 20771 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:08:00.860654 20771 webserver.cc:460] Webserver started at http://127.7.209.126:40221/ using document root <none> and password file <none>
I20260929 03:08:00.861224 20771 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:00.861295 20771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 03:08:00 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651280, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:08:00.910836 20771 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 03:08:00.913672 20787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:00.915369 20771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:00.915508 20771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe"
format_stamp: "Formatted at 2026-09-29 03:07:58 on dist-test-slave-bdps"
server_key: "b3fd1e007adc78606cccba02ab55666c"
server_key_iv: "34b4a57a769e9ca097c6ed800942e3cc"
server_key_version: "kuduclusterkey@0"
I20260929 03:08:00.916168 20771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:00.925959 20771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:00.928836 20771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:00.929009 20771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:00.936739 20771 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:42605
I20260929 03:08:00.936762 20839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:42605 every 8 connection(s)
I20260929 03:08:00.937920 20771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:08:00.942761 20840 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:08:00.944550 20840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe: Bootstrap starting.
I20260929 03:08:00.945230 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 20771
I20260929 03:08:00.945842 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.209.84:43621/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:44543
--local_ip_for_outbound_sockets=127.7.209.65
--tserver_master_addrs=127.7.209.126:42605
--webserver_port=38163
--webserver_interface=127.7.209.65
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.209.84:43621/kms with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:08:00.964149 20840 log.cc:824] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:00.972293 20840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:08:00.972720 20840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe: Bootstrap complete.
I20260929 03:08:00.977099 20840 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42605 } }
I20260929 03:08:00.977527 20840 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf2ab41766484e4c99ee5d5d7e7cd2fe, State: Initialized, Role: FOLLOWER
I20260929 03:08:00.978113 20840 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42605 } }
I20260929 03:08:00.978256 20840 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:08:00.978325 20840 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:08:00.978423 20840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 1 FOLLOWER]: Advancing to term 2
I20260929 03:08:00.980587 20840 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42605 } }
I20260929 03:08:00.980973 20840 leader_election.cc:302] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf2ab41766484e4c99ee5d5d7e7cd2fe; no voters:
I20260929 03:08:00.981376 20840 leader_election.cc:288] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 03:08:00.981463 20844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 2 FOLLOWER]: Leader election won for term 2
I20260929 03:08:00.981703 20844 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [term 2 LEADER]: Becoming Leader. State: Replica: bf2ab41766484e4c99ee5d5d7e7cd2fe, State: Running, Role: LEADER
I20260929 03:08:00.982031 20844 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42605 } }
I20260929 03:08:00.982851 20840 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:08:00.983764 20845 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42605 } } }
I20260929 03:08:00.983770 20846 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf2ab41766484e4c99ee5d5d7e7cd2fe. Latest consensus state: current_term: 2 leader_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf2ab41766484e4c99ee5d5d7e7cd2fe" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42605 } } }
I20260929 03:08:00.983912 20846 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:00.983911 20845 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:00.985783 20840 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 03:08:00.986423 20852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:08:00.993933 20852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:08:00.994731 20852 catalog_manager.cc:1259] Loaded cluster ID: 25b9c7e121164fd5a236d68362b6777b
I20260929 03:08:00.994803 20852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:08:00.997632 20852 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:08:00.998456 20852 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe: Loaded TSK: 0
I20260929 03:08:01.000172 20852 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 03:08:01.115064 20842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:01.115549 20842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:01.115639 20842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:01.122225 20842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:01.122313 20842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:01.122404 20842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:08:01.130779 20842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.209.84:43621/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:44543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=38163
--tserver_master_addrs=127.7.209.126:42605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:01.132498 20842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:01.134181 20842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:01.149277 20877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:01.149477 20880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:01.149358 20878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:01.151746 20842 server_base.cc:1034] running on GCE node
I20260929 03:08:01.152442 20842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:01.153337 20842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:01.160707 20842 hybrid_clock.cc:648] HybridClock initialized: now 1790651281160654 us; error 42 us; skew 500 ppm
Sep 29 03:08:01 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651281, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:01.164494 20842 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:08:01.165673 20842 webserver.cc:460] Webserver started at http://127.7.209.65:38163/ using document root <none> and password file <none>
I20260929 03:08:01.166354 20842 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:01.166440 20842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 03:08:01 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651281, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:08:01.191174 20842 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.004s sys 0.000s
I20260929 03:08:01.194451 20888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:01.196449 20842 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260929 03:08:01.196693 20842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba"
format_stamp: "Formatted at 2026-09-29 03:07:59 on dist-test-slave-bdps"
server_key: "9b4dbb8aae096e3c4177dd8eaf18821a"
server_key_iv: "c6690618dd7f6469451d58b71078f407"
server_key_version: "kuduclusterkey@0"
I20260929 03:08:01.197492 20842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:01.220212 20842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:01.223980 20842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:01.224210 20842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:01.225016 20842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:01.226171 20842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:01.226223 20842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:01.226266 20842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:01.226281 20842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:01.238313 21001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:44543 every 8 connection(s)
I20260929 03:08:01.238287 20842 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:44543
I20260929 03:08:01.239603 20842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:08:01.245406 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 20842
I20260929 03:08:01.245994 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.209.84:43621/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:44301
--local_ip_for_outbound_sockets=127.7.209.66
--tserver_master_addrs=127.7.209.126:42605
--webserver_port=38159
--webserver_interface=127.7.209.66
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.209.84:43621/kms with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:08:01 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651281, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:01.269686 21006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:01.249180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:36887 (local address 127.7.209.126:42605)
0929 03:08:01.250308 (+ 1128us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:01.250334 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:01.251701 (+ 1367us) server_negotiation.cc:407] Connection header received
0929 03:08:01.252518 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:01.252534 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:01.253000 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:01.253408 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:01.255396 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.256901 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.258607 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.259367 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.259543 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:01.262801 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:01.262807 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:01.262825 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:01.262860 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:01.265152 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:01.265904 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:01.265912 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:01.265919 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:01.266100 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:01.266509 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:01.266513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:01.266515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:01.266966 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:01.267112 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:01.267842 (+ 730us) server_negotiation.cc:299] Negotiation successful
0929 03:08:01.268234 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":124,"threads_started":1}
I20260929 03:08:01.272301 21004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:01.247422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42605 (local address 127.7.209.65:36887)
0929 03:08:01.250672 (+ 3250us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:01.250737 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:01.251947 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:01.254098 (+ 2151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:01.254108 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:01.254492 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:01.255186 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:01.255209 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.257062 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.257065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:01.258469 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:01.258480 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.259820 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.259824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:01.260024 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:01.260721 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:01.260743 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:01.262555 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:01.265295 (+ 2740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:01.265302 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:01.265319 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:01.265744 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:01.266228 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:01.266231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:01.266234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:01.266394 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:01.267092 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:01.267100 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:01.267400 (+ 300us) client_negotiation.cc:769] Sending connection context
0929 03:08:01.270828 (+ 3428us) client_negotiation.cc:240] Negotiation successful
0929 03:08:01.271216 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2979,"thread_start_us":181,"threads_started":1}
I20260929 03:08:01.276836 21002 heartbeater.cc:347] Connected to a master server at 127.7.209.126:42605
I20260929 03:08:01.277173 21002 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:01.277987 21002 heartbeater.cc:511] Master 127.7.209.126:42605 requested a full tablet report, sending...
W20260929 03:08:01.280829 21002 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:42605 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 03:08:01.280972 21002 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:01.281184 21002 heartbeater.cc:511] Master 127.7.209.126:42605 requested a full tablet report, sending...
I20260929 03:08:01.282295 21002 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:01.282475 21002 heartbeater.cc:511] Master 127.7.209.126:42605 requested a full tablet report, sending...
W20260929 03:08:01.283365 21002 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 03:08:01.440047 21005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:01.440555 21005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:01.440639 21005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:01.446895 21005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:01.446987 21005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:01.447082 21005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:08:01.455440 21005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.209.84:43621/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:44301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=38159
--tserver_master_addrs=127.7.209.126:42605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:01.457212 21005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:01.459069 21005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:01.472656 21014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:01.472723 21012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:01.473125 21011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:01.473464 21005 server_base.cc:1034] running on GCE node
I20260929 03:08:01.474062 21005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:01.474967 21005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:01.476238 21005 hybrid_clock.cc:648] HybridClock initialized: now 1790651281476208 us; error 44 us; skew 500 ppm
Sep 29 03:08:01 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651281, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:01.479679 21005 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:08:01.480774 21005 webserver.cc:460] Webserver started at http://127.7.209.66:38159/ using document root <none> and password file <none>
I20260929 03:08:01.481464 21005 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:01.481559 21005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 03:08:01 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651281, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:08:01.517113 21005 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.004s sys 0.000s
I20260929 03:08:01.520710 21023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:01.522330 21005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:01.522451 21005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "8b819b06846945ec8cff5e23bdc54ced"
format_stamp: "Formatted at 2026-09-29 03:08:00 on dist-test-slave-bdps"
server_key: "27b34787ce8962aa275e02336ade76e2"
server_key_iv: "76eb901124fbc9cf97950e1dc40b3dd4"
server_key_version: "kuduclusterkey@0"
I20260929 03:08:01.523139 21005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:01.536118 21005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:01.539287 21005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:01.539477 21005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:01.540127 21005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:01.541347 21005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:01.541473 21005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:01.541592 21005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:01.541685 21005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:01.553691 21005 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:44301
I20260929 03:08:01.555243 21005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:08:01.553691 21136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:44301 every 8 connection(s)
I20260929 03:08:01.563720 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 21005
I20260929 03:08:01.564246 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.209.84:43621/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:34945
--local_ip_for_outbound_sockets=127.7.209.67
--tserver_master_addrs=127.7.209.126:42605
--webserver_port=36913
--webserver_interface=127.7.209.67
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:44123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.209.84:43621/kms with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:08:01 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651281, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:01.572261 21006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:01.557474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:49941 (local address 127.7.209.126:42605)
0929 03:08:01.557694 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:01.557699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:01.558755 (+ 1056us) server_negotiation.cc:407] Connection header received
0929 03:08:01.560042 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:01.560044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:01.560101 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:01.560180 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:01.561777 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.562443 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.564162 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.564505 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.564610 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:01.567847 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:01.567849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:01.567852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:01.567879 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:01.569546 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:01.570212 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:01.570216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:01.570218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:01.570327 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:01.570716 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:01.570720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:01.570721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:01.570979 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:01.571070 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:01.571867 (+ 797us) server_negotiation.cc:299] Negotiation successful
0929 03:08:01.572038 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:08:01.574563 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:01.557993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42605 (local address 127.7.209.66:49941)
0929 03:08:01.558547 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:01.558613 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:01.559800 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:01.560411 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:01.560431 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:01.560820 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:01.561558 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:01.561578 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.562561 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.562565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:01.564030 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:01.564042 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.564825 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.564829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:01.565055 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:01.565779 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:01.565801 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:01.567623 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:01.569722 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:01.569728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:01.569749 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:01.570097 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:01.570428 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:01.570431 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:01.570433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:01.570611 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:01.571146 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:01.571153 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:01.571542 (+ 389us) client_negotiation.cc:769] Sending connection context
0929 03:08:01.572725 (+ 1183us) client_negotiation.cc:240] Negotiation successful
0929 03:08:01.573163 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":136,"threads_started":1}
I20260929 03:08:01.576500 21137 heartbeater.cc:347] Connected to a master server at 127.7.209.126:42605
I20260929 03:08:01.576812 21137 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:01.577558 21137 heartbeater.cc:511] Master 127.7.209.126:42605 requested a full tablet report, sending...
W20260929 03:08:01.579887 21137 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:42605 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 03:08:01.580057 21137 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:01.580283 21137 heartbeater.cc:511] Master 127.7.209.126:42605 requested a full tablet report, sending...
I20260929 03:08:01.581162 21137 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:01.581368 21137 heartbeater.cc:511] Master 127.7.209.126:42605 requested a full tablet report, sending...
W20260929 03:08:01.582233 21137 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 03:08:01.719699 21140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:01.720073 21140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:01.720191 21140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:01.726176 21140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:01.726261 21140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:01.726358 21140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:08:01.733754 21140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:44123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.209.84:43621/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:34945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=36913
--tserver_master_addrs=127.7.209.126:42605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:01.735402 21140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:01.737003 21140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:01.749245 21145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:01.751058 21148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:01.752825 21140 server_base.cc:1034] running on GCE node
W20260929 03:08:01.753065 21146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:01.753811 21140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:01.754877 21140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:01.756213 21140 hybrid_clock.cc:648] HybridClock initialized: now 1790651281756177 us; error 59 us; skew 500 ppm
Sep 29 03:08:01 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651281, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:01.760572 21140 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:08:01.761914 21140 webserver.cc:460] Webserver started at http://127.7.209.67:36913/ using document root <none> and password file <none>
I20260929 03:08:01.762777 21140 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:01.762943 21140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 03:08:01 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651281, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:08:01.782272 21140 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 03:08:01.785732 21156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:01.787797 21140 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:08:01.787964 21140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "7fe67f2a265e4f38939653ac9b021453"
format_stamp: "Formatted at 2026-09-29 03:08:00 on dist-test-slave-bdps"
server_key: "75e052d3d4a686f7fcaeb0ec3f960ad8"
server_key_iv: "dbfaa929dc3be39c9198d5b5aebf860c"
server_key_version: "kuduclusterkey@0"
I20260929 03:08:01.788780 21140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:01.820663 21140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:01.823938 21140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:01.824142 21140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:01.824851 21140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:01.825910 21140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:01.825961 21140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:01.826021 21140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:01.826049 21140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:01.837337 21140 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:34945
I20260929 03:08:01.837373 21276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:34945 every 8 connection(s)
I20260929 03:08:01.838604 21140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:08:01.846344 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 21140
Sep 29 03:08:01 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651281, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:01.855816 21006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:01.840928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:32917 (local address 127.7.209.126:42605)
0929 03:08:01.841083 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:01.841087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:01.842204 (+ 1117us) server_negotiation.cc:407] Connection header received
0929 03:08:01.843336 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:01.843340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:01.843408 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:01.843489 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:01.845083 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.846026 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.848215 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.848552 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.848631 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:01.851675 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:01.851679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:01.851682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:01.851722 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:01.853499 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:01.854090 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:01.854095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:01.854096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:01.854217 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:01.854628 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:01.854630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:01.854631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:01.854825 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:01.854904 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:01.855556 (+ 652us) server_negotiation.cc:299] Negotiation successful
0929 03:08:01.855661 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:08:01.857117 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:01.841451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42605 (local address 127.7.209.67:32917)
0929 03:08:01.842021 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:01.842077 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:01.843097 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:01.843705 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:01.843717 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:01.844121 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:01.844884 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:01.844906 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.846206 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.846209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:01.848078 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:01.848088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.848914 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.848918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:01.849132 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:01.849791 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:01.849808 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:01.851500 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:01.853628 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:01.853636 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:01.853653 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:01.853958 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:01.854323 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:01.854326 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:01.854328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:01.854524 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:01.854938 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:01.854945 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:01.855290 (+ 345us) client_negotiation.cc:769] Sending connection context
0929 03:08:01.855509 (+ 219us) client_negotiation.cc:240] Negotiation successful
0929 03:08:01.855849 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":183,"threads_started":1}
I20260929 03:08:01.858584 21277 heartbeater.cc:347] Connected to a master server at 127.7.209.126:42605
I20260929 03:08:01.858862 21277 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:01.859511 21277 heartbeater.cc:511] Master 127.7.209.126:42605 requested a full tablet report, sending...
I20260929 03:08:01.860447 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:01.847188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56066 (local address 127.7.209.126:42605)
0929 03:08:01.847956 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:01.847965 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:01.847979 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:08:01.848018 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:01.848021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:01.848067 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:01.848165 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:01.849185 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.850044 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.851030 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:01.851760 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:01.851842 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:01.853456 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:01.853459 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:01.853461 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:01.853505 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:01.858558 (+ 5053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:01.859058 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:01.859062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:01.859064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:01.859160 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:01.859455 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:01.859458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:01.859460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:01.859763 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:01.859997 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:01.860143 (+ 146us) server_negotiation.cc:299] Negotiation successful
0929 03:08:01.860271 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"spinlock_wait_cycles":29568,"thread_start_us":152,"threads_started":1}
I20260929 03:08:01.861135 20804 ts_manager.cc:194] Registered new tserver with Master: 7fe67f2a265e4f38939653ac9b021453 (127.7.209.67:34945)
I20260929 03:08:01.862478 20804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:32917
Sep 29 03:08:02 dist-test-slave-bdps krb5kdc[19808](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790651282, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:02.283593 21002 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:02.283977 21002 heartbeater.cc:511] Master 127.7.209.126:42605 requested a full tablet report, sending...
I20260929 03:08:02.284845 20804 ts_manager.cc:194] Registered new tserver with Master: a1c5e4cfb1eb4c96b1060ddd97a123ba (127.7.209.65:44543)
I20260929 03:08:02.285708 20804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:36887
I20260929 03:08:02.582530 21137 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:02.582978 21137 heartbeater.cc:511] Master 127.7.209.126:42605 requested a full tablet report, sending...
I20260929 03:08:02.584194 20804 ts_manager.cc:194] Registered new tserver with Master: 8b819b06846945ec8cff5e23bdc54ced (127.7.209.66:44301)
I20260929 03:08:02.585141 20804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:49941
I20260929 03:08:02.587575 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:08:02.603569 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:02.590963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56082 (local address 127.7.209.126:42605)
0929 03:08:02.591816 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:02.591821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:02.592092 (+ 271us) server_negotiation.cc:407] Connection header received
0929 03:08:02.592518 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:02.592521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:02.592588 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:02.592683 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:02.593543 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.594456 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.595705 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.596090 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.596206 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:02.598196 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:02.598200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:02.598203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:02.598234 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:02.601500 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:02.602022 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:02.602026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:02.602028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:02.602120 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:02.602371 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:02.602375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:02.602377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:02.602585 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:02.602721 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:02.603042 (+ 321us) server_negotiation.cc:299] Negotiation successful
0929 03:08:02.603365 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":101,"threads_started":1}
I20260929 03:08:02.607487 20804 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:08:02.612731 20804 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Sep 29 03:08:02 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651282, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:08:02.649323 21298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:02.632734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:44301 (local address 127.0.0.1:44774)
0929 03:08:02.633456 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:02.633521 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:02.633695 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:02.644765 (+ 11070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:02.644769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:02.644786 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:02.644900 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:02.644906 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.646890 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.646894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:02.647969 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:02.647978 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.648997 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.649000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:02.649090 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:02.649111 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 03:08:02.649168 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 03:08:02.649228 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":158,"threads_started":1}
I20260929 03:08:02.649296 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:02.632744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:44543 (local address 127.0.0.1:56060)
0929 03:08:02.642882 (+ 10138us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:02.642896 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:02.642992 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:02.643832 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:02.643839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:02.643864 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:02.644006 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:02.644012 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.645469 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.645473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:02.647381 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:02.647392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.648617 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.648621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:02.648770 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:02.648824 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 03:08:02.648924 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 03:08:02.649110 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10072,"mutex_wait_us":1623,"thread_start_us":104,"threads_started":1}
I20260929 03:08:02.649892 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:02.634437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44774 (local address 127.7.209.66:44301)
0929 03:08:02.644330 (+ 9893us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:02.644338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:02.644361 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:08:02.644439 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:02.644443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:02.644615 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:02.645037 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:02.645215 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.646756 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.648108 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.648879 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.649390 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:02.649566 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:02.649644 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 03:08:02.649746 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9762,"thread_start_us":86,"threads_started":1}
I20260929 03:08:02.650024 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:02.633926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56060 (local address 127.7.209.65:44543)
0929 03:08:02.642725 (+ 8799us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:02.642734 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:02.643093 (+ 359us) server_negotiation.cc:407] Connection header received
0929 03:08:02.643205 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:02.643211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:02.643453 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:02.643620 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:02.644137 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.645340 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.647536 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.648479 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.649566 (+ 1087us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:02.649751 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:02.649797 (+ 46us) server_negotiation.cc:299] Negotiation successful
0929 03:08:02.649879 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8659,"thread_start_us":148,"threads_started":1}
I20260929 03:08:02.654167 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:02.633931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.7.209.67:34945)
0929 03:08:02.637865 (+ 3934us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:02.637874 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:02.641718 (+ 3844us) server_negotiation.cc:407] Connection header received
0929 03:08:02.641838 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:02.641845 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:02.642139 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:02.642338 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:02.647027 (+ 4689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.648233 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.651998 (+ 3765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.652782 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.652862 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:02.653002 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:02.653882 (+ 880us) server_negotiation.cc:299] Negotiation successful
0929 03:08:02.654005 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3800,"thread_start_us":136,"threads_started":1}
I20260929 03:08:02.653590 20918 tablet_service.cc:1467] Processing CreateTablet for tablet 5e31112a876f4c2b84c6b3d762073073 (DEFAULT_TABLE table=test-table [id=5586291eb5e847309aec3fe21ddbd693]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:02.653594 21053 tablet_service.cc:1467] Processing CreateTablet for tablet 5e31112a876f4c2b84c6b3d762073073 (DEFAULT_TABLE table=test-table [id=5586291eb5e847309aec3fe21ddbd693]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:02.654892 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:02.633042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:34945 (local address 127.0.0.1:51984)
0929 03:08:02.638947 (+ 5905us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:02.638963 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:02.639201 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:02.646750 (+ 7549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:02.646754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:02.646774 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:02.646883 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:02.646889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.648386 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.648389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:02.651846 (+ 3457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:02.651857 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.653584 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.653588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:02.653721 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:02.653744 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 03:08:02.654682 (+ 938us) client_negotiation.cc:240] Negotiation successful
0929 03:08:02.654759 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5829,"mutex_wait_us":524,"thread_start_us":97,"threads_started":1}
I20260929 03:08:02.654964 20918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e31112a876f4c2b84c6b3d762073073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:02.654963 21053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e31112a876f4c2b84c6b3d762073073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:02.658046 21211 tablet_service.cc:1467] Processing CreateTablet for tablet 5e31112a876f4c2b84c6b3d762073073 (DEFAULT_TABLE table=test-table [id=5586291eb5e847309aec3fe21ddbd693]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:02.659231 21211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e31112a876f4c2b84c6b3d762073073. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:02.669541 21303 tablet_bootstrap.cc:492] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453: Bootstrap starting.
I20260929 03:08:02.672487 21303 tablet_bootstrap.cc:654] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:02.673507 21303 log.cc:824] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:02.674394 21304 tablet_bootstrap.cc:492] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced: Bootstrap starting.
I20260929 03:08:02.677141 21304 tablet_bootstrap.cc:654] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:02.677341 21303 tablet_bootstrap.cc:492] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453: No bootstrap required, opened a new log
I20260929 03:08:02.677685 21303 ts_tablet_manager.cc:1397] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260929 03:08:02.678226 21304 log.cc:824] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:02.682081 21306 tablet_bootstrap.cc:492] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba: Bootstrap starting.
I20260929 03:08:02.682991 21303 raft_consensus.cc:348] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b819b06846945ec8cff5e23bdc54ced" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44301 } } peers { permanent_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44543 } } peers { permanent_uuid: "7fe67f2a265e4f38939653ac9b021453" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 34945 } }
I20260929 03:08:02.683555 21303 raft_consensus.cc:374] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:02.683656 21303 raft_consensus.cc:729] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fe67f2a265e4f38939653ac9b021453, State: Initialized, Role: FOLLOWER
I20260929 03:08:02.684943 21303 consensus_queue.cc:260] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b819b06846945ec8cff5e23bdc54ced" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44301 } } peers { permanent_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44543 } } peers { permanent_uuid: "7fe67f2a265e4f38939653ac9b021453" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 34945 } }
I20260929 03:08:02.685510 21306 tablet_bootstrap.cc:654] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:02.686527 21303 ts_tablet_manager.cc:1428] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260929 03:08:02.686569 21306 log.cc:824] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:02.687040 21277 heartbeater.cc:503] Master 127.7.209.126:42605 was elected leader, sending a full tablet report...
I20260929 03:08:02.689424 21306 tablet_bootstrap.cc:492] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba: No bootstrap required, opened a new log
I20260929 03:08:02.689673 21306 ts_tablet_manager.cc:1397] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260929 03:08:02.694412 21306 raft_consensus.cc:348] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b819b06846945ec8cff5e23bdc54ced" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44301 } } peers { permanent_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44543 } } peers { permanent_uuid: "7fe67f2a265e4f38939653ac9b021453" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 34945 } }
I20260929 03:08:02.694707 21306 raft_consensus.cc:374] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:02.694795 21306 raft_consensus.cc:729] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1c5e4cfb1eb4c96b1060ddd97a123ba, State: Initialized, Role: FOLLOWER
I20260929 03:08:02.695297 21306 consensus_queue.cc:260] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b819b06846945ec8cff5e23bdc54ced" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44301 } } peers { permanent_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44543 } } peers { permanent_uuid: "7fe67f2a265e4f38939653ac9b021453" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 34945 } }
I20260929 03:08:02.696015 21002 heartbeater.cc:503] Master 127.7.209.126:42605 was elected leader, sending a full tablet report...
I20260929 03:08:02.696497 21306 ts_tablet_manager.cc:1428] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260929 03:08:02.699388 21304 tablet_bootstrap.cc:492] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced: No bootstrap required, opened a new log
I20260929 03:08:02.699658 21304 ts_tablet_manager.cc:1397] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.002s
I20260929 03:08:02.703333 21304 raft_consensus.cc:348] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b819b06846945ec8cff5e23bdc54ced" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44301 } } peers { permanent_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44543 } } peers { permanent_uuid: "7fe67f2a265e4f38939653ac9b021453" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 34945 } }
I20260929 03:08:02.703584 21304 raft_consensus.cc:374] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:02.703617 21304 raft_consensus.cc:729] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b819b06846945ec8cff5e23bdc54ced, State: Initialized, Role: FOLLOWER
I20260929 03:08:02.704057 21304 consensus_queue.cc:260] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b819b06846945ec8cff5e23bdc54ced" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44301 } } peers { permanent_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44543 } } peers { permanent_uuid: "7fe67f2a265e4f38939653ac9b021453" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 34945 } }
I20260929 03:08:02.704749 21137 heartbeater.cc:503] Master 127.7.209.126:42605 was elected leader, sending a full tablet report...
I20260929 03:08:02.705219 21304 ts_tablet_manager.cc:1428] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
W20260929 03:08:02.751401 21003 tablet.cc:2369] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:08:02.807448 21138 tablet.cc:2369] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:08:02.840619 21278 tablet.cc:2369] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:08:02.867887 21311 raft_consensus.cc:482] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:08:02.868275 21311 raft_consensus.cc:504] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b819b06846945ec8cff5e23bdc54ced" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44301 } } peers { permanent_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44543 } } peers { permanent_uuid: "7fe67f2a265e4f38939653ac9b021453" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 34945 } }
I20260929 03:08:02.870206 21311 leader_election.cc:288] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1c5e4cfb1eb4c96b1060ddd97a123ba (127.7.209.65:44543), 7fe67f2a265e4f38939653ac9b021453 (127.7.209.67:34945)
I20260929 03:08:02.876652 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:02.870711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:47015 (local address 127.7.209.65:44543)
0929 03:08:02.871042 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:02.871050 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:02.871368 (+ 318us) server_negotiation.cc:407] Connection header received
0929 03:08:02.871631 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:02.871634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:02.871713 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:02.871775 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:02.872364 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.873217 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.874954 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.875528 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.875620 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:02.875695 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:02.876399 (+ 704us) server_negotiation.cc:299] Negotiation successful
0929 03:08:02.876502 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188}
I20260929 03:08:02.876749 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:02.871633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:39619 (local address 127.7.209.67:34945)
0929 03:08:02.871786 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:02.871791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:02.871997 (+ 206us) server_negotiation.cc:407] Connection header received
0929 03:08:02.872236 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:02.872250 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:02.872310 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:02.872362 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:02.872934 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.873680 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.875363 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.875968 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.876063 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:02.876128 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:02.876518 (+ 390us) server_negotiation.cc:299] Negotiation successful
0929 03:08:02.876609 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:08:02.877342 21312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:02.870486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:44543 (local address 127.7.209.66:47015)
0929 03:08:02.871251 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:02.871269 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:02.871471 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:02.871967 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:02.871972 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:02.871996 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:02.872199 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:02.872208 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.873369 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.873373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:02.874807 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:02.874818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.876108 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.876111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:02.876252 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:02.876278 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 03:08:02.876985 (+ 707us) client_negotiation.cc:240] Negotiation successful
0929 03:08:02.877071 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":50,"spinlock_wait_cycles":4736,"thread_start_us":245,"threads_started":1}
I20260929 03:08:02.877738 21313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:02.871088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:34945 (local address 127.7.209.66:39619)
0929 03:08:02.871875 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:02.871890 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:02.872099 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:02.872511 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:02.872514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:02.872532 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:02.872793 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:02.872802 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.873827 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.873830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:02.875213 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:02.875224 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.876228 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.876232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:02.876373 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:02.876396 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 03:08:02.877198 (+ 802us) client_negotiation.cc:240] Negotiation successful
0929 03:08:02.877291 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":6,"spinlock_wait_cycles":896,"thread_start_us":164,"threads_started":1}
I20260929 03:08:02.878409 20942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e31112a876f4c2b84c6b3d762073073" candidate_uuid: "8b819b06846945ec8cff5e23bdc54ced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba" is_pre_election: true
I20260929 03:08:02.878698 21231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e31112a876f4c2b84c6b3d762073073" candidate_uuid: "8b819b06846945ec8cff5e23bdc54ced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fe67f2a265e4f38939653ac9b021453" is_pre_election: true
I20260929 03:08:02.878835 20942 raft_consensus.cc:2454] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b819b06846945ec8cff5e23bdc54ced in term 0.
I20260929 03:08:02.879014 21231 raft_consensus.cc:2454] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b819b06846945ec8cff5e23bdc54ced in term 0.
I20260929 03:08:02.879487 21026 leader_election.cc:302] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b819b06846945ec8cff5e23bdc54ced, a1c5e4cfb1eb4c96b1060ddd97a123ba; no voters:
I20260929 03:08:02.879889 21311 raft_consensus.cc:2781] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:08:02.879985 21311 raft_consensus.cc:482] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:08:02.880028 21311 raft_consensus.cc:3037] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:02.882207 21311 raft_consensus.cc:504] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b819b06846945ec8cff5e23bdc54ced" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44301 } } peers { permanent_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44543 } } peers { permanent_uuid: "7fe67f2a265e4f38939653ac9b021453" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 34945 } }
I20260929 03:08:02.882707 21311 leader_election.cc:288] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [CANDIDATE]: Term 1 election: Requested vote from peers a1c5e4cfb1eb4c96b1060ddd97a123ba (127.7.209.65:44543), 7fe67f2a265e4f38939653ac9b021453 (127.7.209.67:34945)
I20260929 03:08:02.883309 20942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e31112a876f4c2b84c6b3d762073073" candidate_uuid: "8b819b06846945ec8cff5e23bdc54ced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba"
I20260929 03:08:02.883306 21231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e31112a876f4c2b84c6b3d762073073" candidate_uuid: "8b819b06846945ec8cff5e23bdc54ced" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fe67f2a265e4f38939653ac9b021453"
I20260929 03:08:02.883445 20942 raft_consensus.cc:3037] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:02.883468 21231 raft_consensus.cc:3037] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:02.885015 21231 raft_consensus.cc:2454] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b819b06846945ec8cff5e23bdc54ced in term 1.
I20260929 03:08:02.885260 20942 raft_consensus.cc:2454] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b819b06846945ec8cff5e23bdc54ced in term 1.
I20260929 03:08:02.885444 21027 leader_election.cc:302] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fe67f2a265e4f38939653ac9b021453, 8b819b06846945ec8cff5e23bdc54ced; no voters:
I20260929 03:08:02.885695 21311 raft_consensus.cc:2781] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:08:02.885977 21311 raft_consensus.cc:686] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 1 LEADER]: Becoming Leader. State: Replica: 8b819b06846945ec8cff5e23bdc54ced, State: Running, Role: LEADER
I20260929 03:08:02.886435 21311 consensus_queue.cc:237] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b819b06846945ec8cff5e23bdc54ced" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44301 } } peers { permanent_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44543 } } peers { permanent_uuid: "7fe67f2a265e4f38939653ac9b021453" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 34945 } }
I20260929 03:08:02.891050 20804 catalog_manager.cc:5495] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b819b06846945ec8cff5e23bdc54ced (127.7.209.66). New cstate: current_term: 1 leader_uuid: "8b819b06846945ec8cff5e23bdc54ced" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b819b06846945ec8cff5e23bdc54ced" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fe67f2a265e4f38939653ac9b021453" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 34945 } health_report { overall_health: UNKNOWN } } }
I20260929 03:08:02.936509 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:02.930525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.7.209.66:44301)
0929 03:08:02.930684 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:02.930689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:02.930857 (+ 168us) server_negotiation.cc:407] Connection header received
0929 03:08:02.931048 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:02.931052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:02.931142 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:02.931285 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:08:02.931755 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.933123 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.934415 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:02.934753 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:02.934851 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:02.935249 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:08:02.935967 (+ 718us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:08:02.936096 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:02.936190 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 03:08:02.936259 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 03:08:02.946386 21231 raft_consensus.cc:1261] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453 [term 1 FOLLOWER]: Refusing update from remote peer 8b819b06846945ec8cff5e23bdc54ced: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:08:02.947470 21311 consensus_queue.cc:1035] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fe67f2a265e4f38939653ac9b021453" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:08:02.959441 20942 raft_consensus.cc:1261] T 5e31112a876f4c2b84c6b3d762073073 P a1c5e4cfb1eb4c96b1060ddd97a123ba [term 1 FOLLOWER]: Refusing update from remote peer 8b819b06846945ec8cff5e23bdc54ced: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:08:02.960304 21311 consensus_queue.cc:1035] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1c5e4cfb1eb4c96b1060ddd97a123ba" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Sep 29 03:08:02 dist-test-slave-bdps krb5kdc[19808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651282, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for HTTP/127.7.209.84@KRBTEST.COM
I20260929 03:08:02.970567 21318 mvcc.cc:205] Tried to move back new op lower bound from 7334507654936739840 to 7334507654706274304. Current Snapshot: MvccSnapshot[applied={T|T < 7334507654936739840}]
I20260929 03:08:02.976004 21323 mvcc.cc:205] Tried to move back new op lower bound from 7334507654936739840 to 7334507654706274304. Current Snapshot: MvccSnapshot[applied={T|T < 7334507654936739840}]
I20260929 03:08:02.982712 21319 mvcc.cc:205] Tried to move back new op lower bound from 7334507654936739840 to 7334507654706274304. Current Snapshot: MvccSnapshot[applied={T|T < 7334507654936739840}]
I20260929 03:08:02.997046 20804 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:08:02.997305 20804 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:08:03.001168 20804 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf2ab41766484e4c99ee5d5d7e7cd2fe: Sending DeleteTablet for 3 replicas of tablet 5e31112a876f4c2b84c6b3d762073073
I20260929 03:08:03.002653 21053 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e31112a876f4c2b84c6b3d762073073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:02 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:44774
I20260929 03:08:03.003083 20918 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e31112a876f4c2b84c6b3d762073073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:02 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:56060
I20260929 03:08:03.003434 21211 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e31112a876f4c2b84c6b3d762073073 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:02 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:51984
I20260929 03:08:03.004676 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 20842
I20260929 03:08:03.009850 21330 tablet_replica.cc:330] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453: stopping tablet replica
I20260929 03:08:03.010324 21330 raft_consensus.cc:2229] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:08:03.010699 21330 raft_consensus.cc:2258] T 5e31112a876f4c2b84c6b3d762073073 P 7fe67f2a265e4f38939653ac9b021453 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:03.017920 21328 tablet_replica.cc:330] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced: stopping tablet replica
I20260929 03:08:03.018380 21328 raft_consensus.cc:2229] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:08:03.019007 21328 raft_consensus.cc:2258] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:03.020884 21328 ts_tablet_manager.cc:1905] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:08:03.024916 21328 ts_tablet_manager.cc:1918] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:08:03.025038 21328 log.cc:1196] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/5e31112a876f4c2b84c6b3d762073073
I20260929 03:08:03.025416 21328 ts_tablet_manager.cc:1939] T 5e31112a876f4c2b84c6b3d762073073 P 8b819b06846945ec8cff5e23bdc54ced: Deleting consensus metadata
W20260929 03:08:03.026701 20790 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:44543 (error 108)
I20260929 03:08:03.027434 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 21005
I20260929 03:08:03.028254 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:03.027955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:44543 (local address 127.0.0.1:56076)
0929 03:08:03.028078 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:03.028155 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:44543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 03:08:03.028494 20790 catalog_manager.cc:4584] TS a1c5e4cfb1eb4c96b1060ddd97a123ba (127.7.209.65:44543): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e31112a876f4c2b84c6b3d762073073: Network error: Client connection negotiation failed: client connection to 127.7.209.65:44543: connect: Connection refused (error 111)
I20260929 03:08:03.036979 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 21140
I20260929 03:08:03.043388 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:03.042891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:44301 (local address 127.0.0.1:44786)
0929 03:08:03.043141 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:03.043272 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.66:44301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260929 03:08:03.045334 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 20771
I20260929 03:08:03.055439 8005 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 03:08:03.399993 8005 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 03:08:03.400137 8005 mini_ranger.cc:66] Stopping Ranger...
I20260929 03:08:03.735097 8005 mini_ranger.cc:68] Stopped Ranger
2026-09-29 03:08:03.735 UTC [19838] LOG: received smart shutdown request
2026-09-29 03:08:03.738 UTC [19838] LOG: background worker "logical replication launcher" (PID 19847) exited with exit code 1
2026-09-29 03:08:03.739 UTC [19842] LOG: shutting down
2026-09-29 03:08:03.882 UTC [19838] LOG: database system is shut down
2026-09-29T03:08:03Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49890 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:08:04 dist-test-slave-bdps krb5kdc[21341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:08:04 dist-test-slave-bdps krb5kdc[21341](info): set up 2 sockets
Sep 29 03:08:04 dist-test-slave-bdps krb5kdc[21341](info): commencing operation
krb5kdc: starting...
W20260929 03:08:04.949944 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:08:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:08:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:08:05.074515 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:37831
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:45165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:37831,127.7.209.125:40137,127.7.209.124:35747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:05.200888 21354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:05.201272 21354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:05.201337 21354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:05.205646 21354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:08:05.205713 21354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:05.205741 21354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:05.205763 21354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:08:05.205793 21354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:08:05.211350 21354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:45165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:37831,127.7.209.125:40137,127.7.209.124:35747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:37831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:05.212639 21354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:05.213913 21354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:05.223130 21360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:05.223245 21359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:05.223188 21354 server_base.cc:1034] running on GCE node
W20260929 03:08:05.223143 21362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:05.224125 21354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:05.225414 21354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:05.226567 21354 hybrid_clock.cc:648] HybridClock initialized: now 1790651285226541 us; error 69 us; skew 500 ppm
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:05.229790 21354 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:08:05.230839 21354 webserver.cc:460] Webserver started at http://127.7.209.126:35765/ using document root <none> and password file <none>
I20260929 03:08:05.231475 21354 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:05.231539 21354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:05.231748 21354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:08:05.234203 21354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "87387e28019c4a0c91865c343f4064f0"
format_stamp: "Formatted at 2026-09-29 03:08:05 on dist-test-slave-bdps"
server_key: "17d9c57d6441d004e0a345a7187776df"
server_key_iv: "40215f5e7472bf962f784b5e9284c48e"
server_key_version: "encryptionkey@0"
I20260929 03:08:05.234866 21354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "87387e28019c4a0c91865c343f4064f0"
format_stamp: "Formatted at 2026-09-29 03:08:05 on dist-test-slave-bdps"
server_key: "17d9c57d6441d004e0a345a7187776df"
server_key_iv: "40215f5e7472bf962f784b5e9284c48e"
server_key_version: "encryptionkey@0"
I20260929 03:08:05.239141 21354 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:08:05.241803 21369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:05.243160 21354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:05.243278 21354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "87387e28019c4a0c91865c343f4064f0"
format_stamp: "Formatted at 2026-09-29 03:08:05 on dist-test-slave-bdps"
server_key: "17d9c57d6441d004e0a345a7187776df"
server_key_iv: "40215f5e7472bf962f784b5e9284c48e"
server_key_version: "encryptionkey@0"
I20260929 03:08:05.243373 21354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:05.257942 21354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:05.260851 21354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:05.261025 21354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:05.269037 21354 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:37831
I20260929 03:08:05.269059 21421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:37831 every 8 connection(s)
I20260929 03:08:05.270249 21354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:08:05.272293 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 21354
I20260929 03:08:05.272425 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:08:05.272734 8005 external_mini_cluster.cc:1428] Setting key 3df3ef574e6bfa2eca896f8d325d5cf5
I20260929 03:08:05.273857 21422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:05.278446 21422 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.209.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.125@KRBTEST.COM" created.
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:05.294950 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.282295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.7.209.126:37831)
0929 03:08:05.282585 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.282596 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.282640 (+ 44us) server_negotiation.cc:407] Connection header received
0929 03:08:05.283191 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.283197 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.283460 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.283656 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.284594 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.286029 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.287052 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.287464 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.287613 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.290133 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.290141 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.290155 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.290181 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.292026 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.292479 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.292485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.292486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.292594 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.292854 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.292857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.292858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.293286 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.293394 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.293767 (+ 373us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.293991 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
I20260929 03:08:05.294950 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.281207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:37831 (local address 127.0.0.1:49390)
0929 03:08:05.281818 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.281878 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:05.282890 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:05.283650 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:05.283654 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:05.284145 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:05.284442 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.284457 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.286149 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.286151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.286944 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.286953 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.287607 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.287610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.287719 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.288419 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:05.288437 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:05.289989 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:05.292123 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.292131 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.292140 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.292378 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.292667 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.292670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.292671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.292785 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.293377 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:05.293382 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:05.293517 (+ 135us) client_negotiation.cc:769] Sending connection context
0929 03:08:05.293713 (+ 196us) client_negotiation.cc:240] Negotiation successful
0929 03:08:05.294078 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":78,"thread_start_us":130,"threads_started":1}
I20260929 03:08:05.297700 21422 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:08:05.298436 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.298200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:40137 (local address 127.0.0.1:39556)
0929 03:08:05.298310 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.298384 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 03:08:05.298620 21371 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.209.125:40137: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.125.keytab.
Entry for principal kudu/127.7.209.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.125.keytab.
I20260929 03:08:05.299569 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.299377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:40137 (local address 127.0.0.1:39562)
0929 03:08:05.299475 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.299521 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 03:08:05.299706 21422 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.125:40137: Network error: Client connection negotiation failed: client connection to 127.7.209.125:40137: connect: Connection refused (error 111)
I20260929 03:08:05.300073 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.125:40137
--webserver_interface=127.7.209.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.125.keytab
--principal=kudu/127.7.209.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:45165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:37831,127.7.209.125:40137,127.7.209.124:35747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:08:05.348951 21422 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } attempt: 1
I20260929 03:08:05.350107 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.349715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:40137 (local address 127.0.0.1:39576)
0929 03:08:05.349901 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.349990 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 03:08:05.350875 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.350648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:40137 (local address 127.0.0.1:39586)
0929 03:08:05.350747 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.350802 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 03:08:05.351001 21422 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.125:40137: Network error: Client connection negotiation failed: client connection to 127.7.209.125:40137: connect: Connection refused (error 111)
I20260929 03:08:05.419207 21422 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } attempt: 2
I20260929 03:08:05.420262 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.419856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:40137 (local address 127.0.0.1:39596)
0929 03:08:05.420044 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.420128 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 03:08:05.421192 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.420954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:40137 (local address 127.0.0.1:39608)
0929 03:08:05.421080 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.421135 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:40137: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 03:08:05.421355 21422 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.125:40137: Network error: Client connection negotiation failed: client connection to 127.7.209.125:40137: connect: Connection refused (error 111)
W20260929 03:08:05.425457 21429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:05.425798 21429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:05.425873 21429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:05.430006 21429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:08:05.430068 21429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:05.430095 21429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:05.430127 21429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:08:05.430155 21429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:08:05.435734 21429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:45165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:37831,127.7.209.125:40137,127.7.209.124:35747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.125.keytab
--principal=kudu/127.7.209.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.125:40137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.209.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:05.437054 21429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:05.438344 21429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:05.447639 21437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:05.447639 21435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:05.447819 21429 server_base.cc:1034] running on GCE node
W20260929 03:08:05.448031 21434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:05.448637 21429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:05.449892 21429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:05.451054 21429 hybrid_clock.cc:648] HybridClock initialized: now 1790651285451041 us; error 26 us; skew 500 ppm
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:05.453943 21429 init.cc:377] Logged in from keytab as kudu/127.7.209.125@KRBTEST.COM (short username kudu)
I20260929 03:08:05.454888 21429 webserver.cc:460] Webserver started at http://127.7.209.125:46017/ using document root <none> and password file <none>
I20260929 03:08:05.455464 21429 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:05.455521 21429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:05.455711 21429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:08:05.458035 21429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/data/instance:
uuid: "06e27ef35bbd4bed95379ee55e39d63c"
format_stamp: "Formatted at 2026-09-29 03:08:05 on dist-test-slave-bdps"
server_key: "0e9b04700b3d59afd9cd4dad419425df"
server_key_iv: "2cc2f65b0a64ba07754ab06da1e8926b"
server_key_version: "encryptionkey@0"
I20260929 03:08:05.458583 21429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/wal/instance:
uuid: "06e27ef35bbd4bed95379ee55e39d63c"
format_stamp: "Formatted at 2026-09-29 03:08:05 on dist-test-slave-bdps"
server_key: "0e9b04700b3d59afd9cd4dad419425df"
server_key_iv: "2cc2f65b0a64ba07754ab06da1e8926b"
server_key_version: "encryptionkey@0"
I20260929 03:08:05.462481 21429 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:08:05.464950 21444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:05.466369 21429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:05.466485 21429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/wal
uuid: "06e27ef35bbd4bed95379ee55e39d63c"
format_stamp: "Formatted at 2026-09-29 03:08:05 on dist-test-slave-bdps"
server_key: "0e9b04700b3d59afd9cd4dad419425df"
server_key_iv: "2cc2f65b0a64ba07754ab06da1e8926b"
server_key_version: "encryptionkey@0"
I20260929 03:08:05.466588 21429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:05.499825 21429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:05.503667 21429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:05.503897 21429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:05.512509 21422 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } attempt: 3
I20260929 03:08:05.512759 21429 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.125:40137
I20260929 03:08:05.512764 21496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.125:40137 every 8 connection(s)
I20260929 03:08:05.514554 21429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
I20260929 03:08:05.516955 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 21429
I20260929 03:08:05.517115 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-1/wal/instance
I20260929 03:08:05.517482 8005 external_mini_cluster.cc:1428] Setting key 24b12e5a21177385f3e767876bbe0ff5
I20260929 03:08:05.518611 21498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.125@KRBTEST.COM
I20260929 03:08:05.524320 21498 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:08:05.527541 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.513210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:40137 (local address 127.0.0.1:39610)
0929 03:08:05.513378 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.513400 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:05.513534 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:05.516445 (+ 2911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:05.516448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:05.516687 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:05.516873 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.516882 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.518426 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.518431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.519070 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.519082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.519860 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.519864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.519984 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.520638 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:05.520656 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:05.522473 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:05.525251 (+ 2778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.525254 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.525257 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.525533 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.525971 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.525974 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.525976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.526081 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.526926 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:05.526929 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:05.527060 (+ 131us) client_negotiation.cc:769] Sending connection context
0929 03:08:05.527164 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 03:08:05.527343 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.7.209.124@KRBTEST.COM; defaulting to no policy
I20260929 03:08:05.529253 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.514148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39610 (local address 127.7.209.125:40137)
0929 03:08:05.514659 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.514682 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.514736 (+ 54us) server_negotiation.cc:407] Connection header received
0929 03:08:05.515638 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.515662 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.516095 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.516500 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.517035 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.518284 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.519193 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.519734 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.519897 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.522716 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.522723 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.522745 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.522782 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.525101 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.525669 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.525675 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.525683 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.525864 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.526194 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.526197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.526198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.526739 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.526865 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.527896 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.528259 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.124@KRBTEST.COM" created.
I20260929 03:08:05.531817 21422 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:08:05.532651 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.532362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33590)
0929 03:08:05.532502 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.532566 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.125@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:05.533378 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.533146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33598)
0929 03:08:05.533275 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.533319 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260929 03:08:05.533519 21422 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:35747: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
I20260929 03:08:05.537143 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.526668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49400 (local address 127.7.209.126:37831)
0929 03:08:05.526818 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.526823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.527310 (+ 487us) server_negotiation.cc:407] Connection header received
0929 03:08:05.527498 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.527501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.527551 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.527601 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.528563 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.529918 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.530982 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.531347 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.531447 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.534080 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.534083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.534085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.534105 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.535445 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.535924 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.535926 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.535927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.535990 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.536256 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.536260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.536262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.536452 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.536530 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.536936 (+ 406us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.537038 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 03:08:05.537302 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.526654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:37831 (local address 127.0.0.1:49400)
0929 03:08:05.527181 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.527240 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:05.527401 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:05.527781 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:05.527787 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:05.528260 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:05.528419 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.528426 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.530032 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.530035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.530873 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.530884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.531451 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.531455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.531565 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.532105 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:05.532118 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:05.533957 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:05.535551 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.535560 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.535567 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.535839 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.536065 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.536068 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.536070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.536165 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.536550 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:05.536557 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:05.536699 (+ 142us) client_negotiation.cc:769] Sending connection context
0929 03:08:05.536861 (+ 162us) client_negotiation.cc:240] Negotiation successful
0929 03:08:05.537081 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":178,"threads_started":1}
I20260929 03:08:05.538569 21498 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.125@KRBTEST.COM for kudu/127.7.209.125@KRBTEST.COM
I20260929 03:08:05.546514 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.539062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:40137 (local address 127.0.0.1:39614)
0929 03:08:05.539159 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.539188 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:05.539276 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:05.539484 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:05.539486 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:05.539630 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:05.539700 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.539705 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.540956 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.540959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.541489 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.541497 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.541981 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.541985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.542062 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.542484 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:05.542496 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:05.543662 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:05.545270 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.545272 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.545274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.545448 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.545684 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.545686 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.545687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.545760 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.546079 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:05.546081 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:05.546156 (+ 75us) client_negotiation.cc:769] Sending connection context
0929 03:08:05.546254 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 03:08:05.546381 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260929 03:08:05.546576 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.539108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39614 (local address 127.7.209.125:40137)
0929 03:08:05.539197 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.539201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.539246 (+ 45us) server_negotiation.cc:407] Connection header received
0929 03:08:05.539358 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.539361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.539407 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.539463 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.539794 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.540859 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.541588 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.541884 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.541974 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.543777 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.543780 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.543783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.543812 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.545152 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.545525 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.545527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.545529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.545599 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.545836 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.545839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.545840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.546013 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.546088 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.546291 (+ 203us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.546447 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":40}
I20260929 03:08:05.547247 21498 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.124.keytab.
Entry for principal kudu/127.7.209.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.124.keytab.
I20260929 03:08:05.548022 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.547727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33614)
0929 03:08:05.547857 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.547957 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 03:08:05.548245 21447 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
I20260929 03:08:05.548727 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.124:35747
--webserver_interface=127.7.209.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.124.keytab
--principal=kudu/127.7.209.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:45165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:37831,127.7.209.125:40137,127.7.209.124:35747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:08:05.549125 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.548948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33628)
0929 03:08:05.549033 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.549077 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260929 03:08:05.549249 21498 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:35747: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
I20260929 03:08:05.564685 21422 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } attempt: 1
I20260929 03:08:05.565526 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.565231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33634)
0929 03:08:05.565379 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.565440 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260929 03:08:05.566262 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.566042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33636)
0929 03:08:05.566148 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.566197 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 03:08:05.566381 21422 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:35747: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
I20260929 03:08:05.598572 21498 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } attempt: 1
I20260929 03:08:05.599793 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.599383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33648)
0929 03:08:05.599576 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.599666 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260929 03:08:05.600582 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.600357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33660)
0929 03:08:05.600458 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.600511 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 03:08:05.600724 21498 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:35747: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
I20260929 03:08:05.641538 21422 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } attempt: 2
I20260929 03:08:05.642613 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.642263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33662)
0929 03:08:05.642444 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.642515 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 03:08:05.643501 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.643256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33676)
0929 03:08:05.643367 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.643430 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 03:08:05.643682 21422 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:35747: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
I20260929 03:08:05.669005 21498 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } attempt: 2
I20260929 03:08:05.670105 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.669737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33682)
0929 03:08:05.669937 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.670007 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260929 03:08:05.670919 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.670679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33692)
0929 03:08:05.670784 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.670837 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 03:08:05.671088 21498 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:35747: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
W20260929 03:08:05.676208 21504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:05.676482 21504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:05.676559 21504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:05.680696 21504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:08:05.680755 21504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:05.680783 21504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:05.680817 21504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:08:05.680847 21504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:08:05.686427 21504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:45165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:37831,127.7.209.125:40137,127.7.209.124:35747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.124.keytab
--principal=kudu/127.7.209.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.124:35747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.209.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:05.687820 21504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:05.689055 21504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:05.698011 21510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:05.698019 21509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:05.698016 21512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:05.698067 21504 server_base.cc:1034] running on GCE node
I20260929 03:08:05.698947 21504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:05.700116 21504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:05.701274 21504 hybrid_clock.cc:648] HybridClock initialized: now 1790651285701257 us; error 30 us; skew 500 ppm
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:05.704177 21504 init.cc:377] Logged in from keytab as kudu/127.7.209.124@KRBTEST.COM (short username kudu)
I20260929 03:08:05.705119 21504 webserver.cc:460] Webserver started at http://127.7.209.124:33041/ using document root <none> and password file <none>
I20260929 03:08:05.705837 21504 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:05.705911 21504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:05.706142 21504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:08:05.708530 21504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/data/instance:
uuid: "74c8bb8c719e455e93c348fe88cb3b5e"
format_stamp: "Formatted at 2026-09-29 03:08:05 on dist-test-slave-bdps"
server_key: "93ac9f8acebeeec9e2e9e3f29faacdbd"
server_key_iv: "bbbcf7cf534df6e4cbb75d2c1e74f495"
server_key_version: "encryptionkey@0"
I20260929 03:08:05.709115 21504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/wal/instance:
uuid: "74c8bb8c719e455e93c348fe88cb3b5e"
format_stamp: "Formatted at 2026-09-29 03:08:05 on dist-test-slave-bdps"
server_key: "93ac9f8acebeeec9e2e9e3f29faacdbd"
server_key_iv: "bbbcf7cf534df6e4cbb75d2c1e74f495"
server_key_version: "encryptionkey@0"
I20260929 03:08:05.713128 21504 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 03:08:05.715742 21519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:05.717119 21504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 03:08:05.717252 21504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/wal
uuid: "74c8bb8c719e455e93c348fe88cb3b5e"
format_stamp: "Formatted at 2026-09-29 03:08:05 on dist-test-slave-bdps"
server_key: "93ac9f8acebeeec9e2e9e3f29faacdbd"
server_key_iv: "bbbcf7cf534df6e4cbb75d2c1e74f495"
server_key_version: "encryptionkey@0"
I20260929 03:08:05.717344 21504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:05.742849 21422 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } attempt: 3
I20260929 03:08:05.743746 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.743422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33708)
0929 03:08:05.743572 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.743641 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 03:08:05.744447 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.744237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33710)
0929 03:08:05.744326 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.744381 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 03:08:05.744642 21422 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:35747: Network error: Client connection negotiation failed: client connection to 127.7.209.124:35747: connect: Connection refused (error 111)
I20260929 03:08:05.747532 21504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:05.750762 21504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:05.750926 21504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:05.758668 21504 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.124:35747
I20260929 03:08:05.758709 21571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.124:35747 every 8 connection(s)
I20260929 03:08:05.759752 21504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
I20260929 03:08:05.762233 21498 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } attempt: 3
I20260929 03:08:05.763131 21572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:05.765786 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 21504
I20260929 03:08:05.765913 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790651120485886-8005-0/minicluster-data/master-2/wal/instance
I20260929 03:08:05.766271 8005 external_mini_cluster.cc:1428] Setting key b986b5a0e494c4e3c8c3c9d8b580e797
I20260929 03:08:05.768620 21572 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.125@KRBTEST.COM for kudu/127.7.209.124@KRBTEST.COM
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.124@KRBTEST.COM
I20260929 03:08:05.777251 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.762846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33720)
0929 03:08:05.763011 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.763029 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:05.763154 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:05.765991 (+ 2837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:05.765995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:05.766226 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:05.766376 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.766386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.768169 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.768173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.768729 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.768749 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.769733 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.769737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.769839 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.770434 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:05.770450 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:05.772062 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:05.775141 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.775145 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.775147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.775402 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.775799 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.775803 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.775805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.775896 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.776663 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:05.776666 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:05.776786 (+ 120us) client_negotiation.cc:769] Sending connection context
0929 03:08:05.776881 (+ 95us) client_negotiation.cc:240] Negotiation successful
0929 03:08:05.777038 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.124@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:05.779023 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.763586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.7.209.124:35747)
0929 03:08:05.764024 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.764046 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.764094 (+ 48us) server_negotiation.cc:407] Connection header received
0929 03:08:05.765034 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.765062 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.765537 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.765987 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.766577 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.768051 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.768846 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.769561 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.769754 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.772251 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.772259 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.772273 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.772307 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.775026 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.775532 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.775538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.775544 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.775684 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.775997 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.776001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.776002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.776537 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.777970 (+ 1433us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.778039 (+ 69us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.778141 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":284,"spinlock_wait_cycles":9984,"thread_start_us":126,"threads_started":1}
I20260929 03:08:05.779086 21575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.766954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33732 (local address 127.7.209.124:35747)
0929 03:08:05.767350 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.767359 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.767589 (+ 230us) server_negotiation.cc:407] Connection header received
0929 03:08:05.767727 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.767730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.767777 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.767871 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.768789 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.769419 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.770462 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.770790 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.770890 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.773717 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.773721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.773724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.773750 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.775274 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.775738 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.775743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.775744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.775836 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.776136 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.776139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.776141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.776537 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.776753 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.777484 (+ 731us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.777860 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":335,"thread_start_us":171,"threads_started":1}
I20260929 03:08:05.781473 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.770856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49406 (local address 127.7.209.126:37831)
0929 03:08:05.771120 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.771147 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.771607 (+ 460us) server_negotiation.cc:407] Connection header received
0929 03:08:05.771782 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.771786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.771848 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.771939 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.772715 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.773257 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.774610 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.774852 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.774943 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.778106 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.778110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.778112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.778141 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.779700 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.780157 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.780161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.780163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.780272 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.780567 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.780570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.780571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.780784 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.780867 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.781191 (+ 324us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.781317 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260929 03:08:05.781495 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.770755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:37831 (local address 127.0.0.1:49406)
0929 03:08:05.771472 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.771526 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:05.771653 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:05.772005 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:05.772011 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:05.772418 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:05.772534 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.772539 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.773419 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.773423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.774345 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.774354 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.774956 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.774959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.775069 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.775559 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:05.775573 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:05.777934 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:05.779800 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.779805 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.779809 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.780054 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.780388 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.780391 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.780393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.780483 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.780882 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:05.780888 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:05.781003 (+ 115us) client_negotiation.cc:769] Sending connection context
0929 03:08:05.781145 (+ 142us) client_negotiation.cc:240] Negotiation successful
0929 03:08:05.781328 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":97,"threads_started":1}
I20260929 03:08:05.782861 21572 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:08:05.785754 21498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c: Bootstrap starting.
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.124@KRBTEST.COM for kudu/127.7.209.125@KRBTEST.COM
I20260929 03:08:05.789702 21498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c: Neither blocks nor log segments found. Creating new log.
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:05.790881 21498 log.cc:824] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:05.792613 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.783526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39626 (local address 127.7.209.125:40137)
0929 03:08:05.783668 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.783673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.783872 (+ 199us) server_negotiation.cc:407] Connection header received
0929 03:08:05.783963 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.783966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.784015 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.784099 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.784542 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.785281 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.786370 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.786699 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.786802 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.788920 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.788924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.788927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.788955 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.790386 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.791182 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.791186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.791188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.791294 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.791618 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.791621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.791622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.791885 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.791976 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.792278 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.792432 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 03:08:05.792618 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.783356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:40137 (local address 127.0.0.1:39626)
0929 03:08:05.783807 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.783819 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:05.783892 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:05.784126 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:05.784128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:05.784291 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:05.784427 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.784434 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.785379 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.785382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.786233 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.786250 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.786806 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.786809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.786896 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.787411 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:05.787424 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:05.788778 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:05.790818 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.790821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.790823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.791047 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.791419 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.791422 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.791424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.791522 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.791990 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:05.791994 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:05.792113 (+ 119us) client_negotiation.cc:769] Sending connection context
0929 03:08:05.792229 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 03:08:05.792430 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384}
I20260929 03:08:05.793522 21572 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:08:05.794603 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.783292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.7.209.126:37831)
0929 03:08:05.783402 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.783407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.783619 (+ 212us) server_negotiation.cc:407] Connection header received
0929 03:08:05.783755 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.783759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.783810 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.783882 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.784696 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.785186 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.786928 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.787239 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.787350 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.790544 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.790547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.790550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.790576 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.792200 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.793113 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.793118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.793120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.793232 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.793550 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.793553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.793555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.793865 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.794142 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.794272 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.794427 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 03:08:05.794876 21498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c: No bootstrap required, opened a new log
I20260929 03:08:05.799510 21498 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } }
I20260929 03:08:05.799822 21498 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.124@KRBTEST.COM for kudu/127.7.209.124@KRBTEST.COM
I20260929 03:08:05.799883 21498 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06e27ef35bbd4bed95379ee55e39d63c, State: Initialized, Role: FOLLOWER
I20260929 03:08:05.800496 21498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } }
I20260929 03:08:05.801383 21579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } } }
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.125@KRBTEST.COM
I20260929 03:08:05.801540 21579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:08:05.802361 21498 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:08:05.803785 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.794117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33742)
0929 03:08:05.794274 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.794290 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:05.794367 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:05.794647 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:05.794650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:05.794819 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:05.794939 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.794946 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.795880 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.795884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.797522 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.797533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.798255 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.798257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.798311 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.798771 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:05.798781 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:05.800195 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:05.802114 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.802117 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.802119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.802351 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.802615 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.802618 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.802619 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.802709 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.803213 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:05.803217 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:05.803316 (+ 99us) client_negotiation.cc:769] Sending connection context
0929 03:08:05.803424 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 03:08:05.803555 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 03:08:05.803797 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.794344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33742 (local address 127.7.209.124:35747)
0929 03:08:05.794449 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.794453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.794465 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:08:05.794497 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.794500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.794543 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.794622 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.795036 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.795694 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.797703 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.798060 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.798151 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.800332 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.800335 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.800338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.800359 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.802018 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.802444 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.802447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.802449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.802529 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.802805 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.802808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.802809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.803105 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.803210 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.803496 (+ 286us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.803628 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 03:08:05.807330 21572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e: Bootstrap starting.
W20260929 03:08:05.808286 21590 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:08:05.808348 21590 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:08:05.806576 21429 master_runner.cc:186] Error getting master registration for 127.7.209.126:37831: OK, instance_id { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" instance_seqno: 1790651285266693 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 03:08:05.809573 21429 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 03:08:05.810532 21572 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:05.810608 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.796489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39636 (local address 127.7.209.125:40137)
0929 03:08:05.796597 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.796601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.796613 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:08:05.796649 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.796652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.796694 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.796767 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.797347 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.798022 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.799071 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.799378 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.799468 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.802071 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.802075 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.802078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.802104 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.803677 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.804264 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.804268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.804270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.804350 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.809759 (+ 5409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.809762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.809764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.809963 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.810153 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.810342 (+ 189us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.810459 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 03:08:05.811528 21572 log.cc:824] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:05.813961 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:08:05.814021 21572 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e: No bootstrap required, opened a new log
I20260929 03:08:05.818710 21572 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } }
I20260929 03:08:05.819008 21572 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:05.819072 21572 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74c8bb8c719e455e93c348fe88cb3b5e, State: Initialized, Role: FOLLOWER
I20260929 03:08:05.819640 21572 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } }
I20260929 03:08:05.820197 21598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } } }
I20260929 03:08:05.820353 21598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:08:05.821328 21572 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:08:05.825507 21504 master_runner.cc:186] Error getting master registration for 127.7.209.126:37831: OK, instance_id { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" instance_seqno: 1790651285266693 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 03:08:05.829385 21504 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 03:08:05.833256 21609 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:08:05.833329 21609 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:08:05.836205 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.816161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.7.209.124:35747)
0929 03:08:05.816285 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.816289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.825315 (+ 9026us) server_negotiation.cc:407] Connection header received
0929 03:08:05.825524 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.825528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.825583 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.825676 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.826803 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.827574 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.828928 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.829242 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.829316 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.831438 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.831442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.831444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.831470 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.833432 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.834792 (+ 1360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.834796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.834798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.834886 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.835197 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.835201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.835203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.835413 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.835548 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.835913 (+ 365us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.836031 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 03:08:05.836222 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.824297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49428 (local address 127.7.209.126:37831)
0929 03:08:05.824428 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.824433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.825323 (+ 890us) server_negotiation.cc:407] Connection header received
0929 03:08:05.825575 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.825578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.825662 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.825718 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.826777 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.827611 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.828926 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.829270 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.829342 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.831904 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.831908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.831911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.831938 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.834246 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.834821 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.834826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.834827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.834915 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.835230 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.835234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.835236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.835446 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.835540 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.835911 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.836042 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 03:08:05.841971 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.825079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39644 (local address 127.7.209.125:40137)
0929 03:08:05.829597 (+ 4518us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.829601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.830059 (+ 458us) server_negotiation.cc:407] Connection header received
0929 03:08:05.830154 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.830157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.830212 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.830288 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.831916 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.832694 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.833848 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.834196 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.834269 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.838578 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.838582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.838585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.838614 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.840210 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.840668 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.840671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.840674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.840772 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.841035 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.841038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.841040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.841264 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.841398 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.841652 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.841789 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4413}
I20260929 03:08:05.908818 21422 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } attempt: 4
Sep 29 03:08:05 dist-test-slave-bdps krb5kdc[21341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651285, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.124@KRBTEST.COM
I20260929 03:08:05.919803 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.909424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:35747 (local address 127.0.0.1:33772)
0929 03:08:05.909697 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:05.909716 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:05.909824 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:05.910322 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:05.910326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:05.910584 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:05.910720 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.910726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.911845 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.911849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.912670 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:05.912682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.913255 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.913258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:05.913344 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.914052 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:05.914068 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:05.916050 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:05.918152 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.918156 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.918159 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.918447 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.918706 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:05.918710 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:05.918712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:05.918816 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:05.919223 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:05.919226 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:05.919340 (+ 114us) client_negotiation.cc:769] Sending connection context
0929 03:08:05.919449 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 03:08:05.919606 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260929 03:08:05.919803 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:05.909486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33772 (local address 127.7.209.124:35747)
0929 03:08:05.909998 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:05.910002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:05.910016 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:08:05.910057 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:05.910061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:05.910110 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:05.910178 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:05.910866 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.911696 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.912793 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:05.913138 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:05.913241 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:05.916202 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:05.916207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:05.916210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:05.916237 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:05.918034 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.918537 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.918540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.918542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.918627 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:05.918897 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:05.918899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:05.918901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:05.919125 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:05.919216 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:05.919499 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 03:08:05.919617 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416}
I20260929 03:08:05.924434 21422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0: Bootstrap starting.
I20260929 03:08:05.928179 21422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:05.929275 21422 log.cc:824] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:05.931950 21422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0: No bootstrap required, opened a new log
I20260929 03:08:05.936502 21422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } }
I20260929 03:08:05.936774 21422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:05.936834 21422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87387e28019c4a0c91865c343f4064f0, State: Initialized, Role: FOLLOWER
I20260929 03:08:05.937389 21422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } }
I20260929 03:08:05.938007 21614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } } }
I20260929 03:08:05.938247 21614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:08:05.939064 21422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 03:08:05.946297 21625 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:08:05.946406 21625 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:08:05.952945 21354 master_runner.cc:263] No leader master found from master 87387e28019c4a0c91865c343f4064f0
I20260929 03:08:05.953047 21354 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260929 03:08:06.002395 21525 tablet.cc:2369] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:08:06.005927 21450 tablet.cc:2369] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:08:06.013168 21375 tablet.cc:2369] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:08:06.056320 21614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:08:06.056530 21614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } }
I20260929 03:08:06.057543 21614 leader_election.cc:288] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06e27ef35bbd4bed95379ee55e39d63c (127.7.209.125:40137), 74c8bb8c719e455e93c348fe88cb3b5e (127.7.209.124:35747)
I20260929 03:08:06.058130 21546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87387e28019c4a0c91865c343f4064f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" is_pre_election: true
I20260929 03:08:06.058130 21471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87387e28019c4a0c91865c343f4064f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06e27ef35bbd4bed95379ee55e39d63c" is_pre_election: true
I20260929 03:08:06.058496 21471 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87387e28019c4a0c91865c343f4064f0 in term 0.
I20260929 03:08:06.058497 21546 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87387e28019c4a0c91865c343f4064f0 in term 0.
I20260929 03:08:06.058934 21371 leader_election.cc:302] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06e27ef35bbd4bed95379ee55e39d63c, 87387e28019c4a0c91865c343f4064f0; no voters:
I20260929 03:08:06.059204 21614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:08:06.059300 21614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:08:06.059348 21614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:06.060531 21614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } }
I20260929 03:08:06.060842 21614 leader_election.cc:288] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [CANDIDATE]: Term 1 election: Requested vote from peers 06e27ef35bbd4bed95379ee55e39d63c (127.7.209.125:40137), 74c8bb8c719e455e93c348fe88cb3b5e (127.7.209.124:35747)
I20260929 03:08:06.061131 21546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87387e28019c4a0c91865c343f4064f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74c8bb8c719e455e93c348fe88cb3b5e"
I20260929 03:08:06.061146 21471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87387e28019c4a0c91865c343f4064f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06e27ef35bbd4bed95379ee55e39d63c"
I20260929 03:08:06.061288 21546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:06.061309 21471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:06.062505 21546 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87387e28019c4a0c91865c343f4064f0 in term 1.
I20260929 03:08:06.062793 21372 leader_election.cc:302] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74c8bb8c719e455e93c348fe88cb3b5e, 87387e28019c4a0c91865c343f4064f0; no voters:
I20260929 03:08:06.062861 21471 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87387e28019c4a0c91865c343f4064f0 in term 1.
I20260929 03:08:06.062958 21614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:08:06.063262 21614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [term 1 LEADER]: Becoming Leader. State: Replica: 87387e28019c4a0c91865c343f4064f0, State: Running, Role: LEADER
I20260929 03:08:06.063601 21614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } }
I20260929 03:08:06.065666 21627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87387e28019c4a0c91865c343f4064f0. Latest consensus state: current_term: 1 leader_uuid: "87387e28019c4a0c91865c343f4064f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } } }
I20260929 03:08:06.065805 21627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:06.066247 21629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:08:06.067178 21629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:08:06.073112 21546 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [term 1 FOLLOWER]: Refusing update from remote peer 87387e28019c4a0c91865c343f4064f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:08:06.073364 21471 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [term 1 FOLLOWER]: Refusing update from remote peer 87387e28019c4a0c91865c343f4064f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:08:06.073872 21627 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:08:06.074059 21614 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:08:06.078671 21598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87387e28019c4a0c91865c343f4064f0. Latest consensus state: current_term: 1 leader_uuid: "87387e28019c4a0c91865c343f4064f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } } }
I20260929 03:08:06.078804 21598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:08:06.080163 21579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87387e28019c4a0c91865c343f4064f0. Latest consensus state: current_term: 1 leader_uuid: "87387e28019c4a0c91865c343f4064f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } } }
I20260929 03:08:06.080298 21579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:08:06.082609 21627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87387e28019c4a0c91865c343f4064f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } } }
I20260929 03:08:06.082762 21627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:06.082880 21627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87387e28019c4a0c91865c343f4064f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } } }
I20260929 03:08:06.082983 21627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:06.083038 21598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87387e28019c4a0c91865c343f4064f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } } }
I20260929 03:08:06.083086 21630 mvcc.cc:205] Tried to move back new op lower bound from 7334507667746091008 to 7334507667718930432. Current Snapshot: MvccSnapshot[applied={T|T < 7334507667746091008}]
I20260929 03:08:06.083151 21598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:08:06.083666 21579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87387e28019c4a0c91865c343f4064f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87387e28019c4a0c91865c343f4064f0" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37831 } } peers { permanent_uuid: "06e27ef35bbd4bed95379ee55e39d63c" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 40137 } } peers { permanent_uuid: "74c8bb8c719e455e93c348fe88cb3b5e" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 35747 } } }
I20260929 03:08:06.083788 21579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:08:06.085295 21629 catalog_manager.cc:1347] Generated new cluster ID: 3e628e222c564c5e96398537ebe4f38f
I20260929 03:08:06.085367 21629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:08:06.104692 21629 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:08:06.106379 21629 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:08:06.122259 21629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87387e28019c4a0c91865c343f4064f0: Generated new TSK 0
I20260929 03:08:06.123593 21629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:08:06.810217 21590 catalog_manager.cc:1259] Loaded cluster ID: 3e628e222c564c5e96398537ebe4f38f
I20260929 03:08:06.810335 21590 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c: loading cluster ID for follower catalog manager: success
I20260929 03:08:06.812424 21590 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c: acquiring CA information for follower catalog manager: success
I20260929 03:08:06.813925 21590 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 06e27ef35bbd4bed95379ee55e39d63c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 03:08:06.835078 21609 catalog_manager.cc:1259] Loaded cluster ID: 3e628e222c564c5e96398537ebe4f38f
I20260929 03:08:06.835191 21609 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e: loading cluster ID for follower catalog manager: success
I20260929 03:08:06.837070 21609 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e: acquiring CA information for follower catalog manager: success
I20260929 03:08:06.838521 21609 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 74c8bb8c719e455e93c348fe88cb3b5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 03:08:07.198572 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 21354
I20260929 03:08:07.205535 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 21429
I20260929 03:08:07.211110 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 21504
2026-09-29T03:08:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3221 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:08:07 dist-test-slave-bdps krb5kdc[21645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:08:07 dist-test-slave-bdps krb5kdc[21645](info): set up 2 sockets
Sep 29 03:08:07 dist-test-slave-bdps krb5kdc[21645](info): commencing operation
krb5kdc: starting...
W20260929 03:08:08.174129 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:08:08 dist-test-slave-bdps krb5kdc[21645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:08:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:08:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:08:08.298187 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:45033
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:43217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:45033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:08.420890 21658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:08.421162 21658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:08.421200 21658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:08.425124 21658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:08:08.425180 21658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:08.425197 21658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:08.425212 21658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:08:08.425225 21658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:08:08.430533 21658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:43217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:45033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:45033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:08.431797 21658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:08.433012 21658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:08.442202 21664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:08.442207 21666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:08.442323 21658 server_base.cc:1034] running on GCE node
W20260929 03:08:08.442201 21663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:08.443140 21658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:08.444464 21658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:08.445649 21658 hybrid_clock.cc:648] HybridClock initialized: now 1790651288445619 us; error 40 us; skew 500 ppm
Sep 29 03:08:08 dist-test-slave-bdps krb5kdc[21645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651288, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:08.448616 21658 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:08:08.449589 21658 webserver.cc:460] Webserver started at http://127.7.209.126:33265/ using document root <none> and password file <none>
I20260929 03:08:08.450242 21658 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:08.450313 21658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:08.450542 21658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:08:08.452870 21658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "e64d7dd1003a4dfd8c6d8b30c2fddde7"
format_stamp: "Formatted at 2026-09-29 03:08:08 on dist-test-slave-bdps"
server_key: "f2297986aa38317c1fede34e4cb15bba"
server_key_iv: "2f5a01e3eb2d05ed9178ea1838998a32"
server_key_version: "encryptionkey@0"
I20260929 03:08:08.453483 21658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "e64d7dd1003a4dfd8c6d8b30c2fddde7"
format_stamp: "Formatted at 2026-09-29 03:08:08 on dist-test-slave-bdps"
server_key: "f2297986aa38317c1fede34e4cb15bba"
server_key_iv: "2f5a01e3eb2d05ed9178ea1838998a32"
server_key_version: "encryptionkey@0"
I20260929 03:08:08.457592 21658 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260929 03:08:08.460255 21673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:08.461591 21658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 03:08:08.461755 21658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "e64d7dd1003a4dfd8c6d8b30c2fddde7"
format_stamp: "Formatted at 2026-09-29 03:08:08 on dist-test-slave-bdps"
server_key: "f2297986aa38317c1fede34e4cb15bba"
server_key_iv: "2f5a01e3eb2d05ed9178ea1838998a32"
server_key_version: "encryptionkey@0"
I20260929 03:08:08.461858 21658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:08.483383 21658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:08.487825 21658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:08.488058 21658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:08.496248 21658 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:45033
I20260929 03:08:08.496265 21725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:45033 every 8 connection(s)
I20260929 03:08:08.497406 21658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:08:08.500867 21726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:08.504987 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 21658
I20260929 03:08:08.505110 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:08:08.505407 8005 external_mini_cluster.cc:1428] Setting key d80353ac80121b5635c7c964669b7190
I20260929 03:08:08.507860 21726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7: Bootstrap starting.
I20260929 03:08:08.510944 21726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:08.512002 21726 log.cc:824] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:08.514673 21726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7: No bootstrap required, opened a new log
Sep 29 03:08:08 dist-test-slave-bdps krb5kdc[21645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:08.518206 21726 raft_consensus.cc:348] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64d7dd1003a4dfd8c6d8b30c2fddde7" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45033 } }
I20260929 03:08:08.518489 21726 raft_consensus.cc:374] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:08.518551 21726 raft_consensus.cc:729] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64d7dd1003a4dfd8c6d8b30c2fddde7, State: Initialized, Role: FOLLOWER
I20260929 03:08:08.519057 21726 consensus_queue.cc:260] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64d7dd1003a4dfd8c6d8b30c2fddde7" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45033 } }
I20260929 03:08:08.519204 21726 raft_consensus.cc:388] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:08:08.519276 21726 raft_consensus.cc:482] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:08:08.519387 21726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:08.520426 21726 raft_consensus.cc:504] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64d7dd1003a4dfd8c6d8b30c2fddde7" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45033 } }
I20260929 03:08:08.520823 21726 leader_election.cc:302] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e64d7dd1003a4dfd8c6d8b30c2fddde7; no voters:
I20260929 03:08:08.521159 21726 leader_election.cc:288] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:08:08.521286 21731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:08:08.521479 21731 raft_consensus.cc:686] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [term 1 LEADER]: Becoming Leader. State: Replica: e64d7dd1003a4dfd8c6d8b30c2fddde7, State: Running, Role: LEADER
I20260929 03:08:08.521847 21731 consensus_queue.cc:237] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64d7dd1003a4dfd8c6d8b30c2fddde7" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45033 } }
I20260929 03:08:08.522003 21729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:08.506982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.7.209.126:45033)
0929 03:08:08.507453 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:08.507473 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:08.507519 (+ 46us) server_negotiation.cc:407] Connection header received
0929 03:08:08.508302 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:08.508324 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:08.508745 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:08.509094 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:08.510133 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:08.511354 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:08.512588 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:08.513225 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:08.513353 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:08.516268 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:08.516274 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:08.516291 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:08.516321 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:08.518635 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:08.519105 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:08.519110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:08.519115 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:08.519256 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:08.519531 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:08.519535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:08.519537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:08.519991 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:08.520201 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:08.520488 (+ 287us) server_negotiation.cc:299] Negotiation successful
0929 03:08:08.520868 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":140,"threads_started":1}
I20260929 03:08:08.522716 21726 sys_catalog.cc:564] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:08:08.525768 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:08:08.525815 21733 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e64d7dd1003a4dfd8c6d8b30c2fddde7. Latest consensus state: current_term: 1 leader_uuid: "e64d7dd1003a4dfd8c6d8b30c2fddde7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64d7dd1003a4dfd8c6d8b30c2fddde7" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45033 } } }
I20260929 03:08:08.525935 21733 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:08.525883 21732 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e64d7dd1003a4dfd8c6d8b30c2fddde7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64d7dd1003a4dfd8c6d8b30c2fddde7" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 45033 } } }
I20260929 03:08:08.525978 21732 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7 [sys.catalog]: This master's current role is: LEADER
W20260929 03:08:08.528716 21747 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:08:08.528787 21747 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:08:08.528856 21748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:08:08.529573 21748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:08:08.535421 21748 catalog_manager.cc:1347] Generated new cluster ID: 7bf9e670f0224c21893c3e1ff247e24f
I20260929 03:08:08.535480 21748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:08:08.551710 21748 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:08:08.552755 21748 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:08:08.559471 21748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e64d7dd1003a4dfd8c6d8b30c2fddde7: Generated new TSK 0
I20260929 03:08:08.560256 21748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:08:08.582319 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 21658
2026-09-29T03:08:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1371 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:08:08 dist-test-slave-bdps krb5kdc[21754](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:08:08 dist-test-slave-bdps krb5kdc[21754](info): set up 2 sockets
Sep 29 03:08:08 dist-test-slave-bdps krb5kdc[21754](info): commencing operation
krb5kdc: starting...
W20260929 03:08:09.538058 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:08:09 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:08:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:08:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:08:09.664065 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:42781
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:42781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:09.787721 21767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:09.788056 21767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:09.788097 21767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:09.792198 21767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:08:09.792274 21767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:09.792294 21767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:09.792312 21767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:08:09.792352 21767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:08:09.799183 21767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:42781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:42781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:09.800549 21767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:09.801872 21767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:09.811197 21775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:09.811239 21772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:09.811208 21773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:09.811479 21767 server_base.cc:1034] running on GCE node
I20260929 03:08:09.812163 21767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:09.813462 21767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:09.814617 21767 hybrid_clock.cc:648] HybridClock initialized: now 1790651289814593 us; error 41 us; skew 500 ppm
Sep 29 03:08:09 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651289, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:09.821509 21767 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:08:09.822553 21767 webserver.cc:460] Webserver started at http://127.7.209.126:46451/ using document root <none> and password file <none>
I20260929 03:08:09.823191 21767 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:09.823261 21767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:09.823504 21767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:08:09.825903 21767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "e5bc667ab99d40fe9f4a6f856d2b4937"
format_stamp: "Formatted at 2026-09-29 03:08:09 on dist-test-slave-bdps"
server_key: "25026daffd09a902877af35d5b2b79a2"
server_key_iv: "d8673b20d45817dc6194c79945384c11"
server_key_version: "encryptionkey@0"
I20260929 03:08:09.826524 21767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "e5bc667ab99d40fe9f4a6f856d2b4937"
format_stamp: "Formatted at 2026-09-29 03:08:09 on dist-test-slave-bdps"
server_key: "25026daffd09a902877af35d5b2b79a2"
server_key_iv: "d8673b20d45817dc6194c79945384c11"
server_key_version: "encryptionkey@0"
I20260929 03:08:09.830896 21767 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:08:09.833912 21782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:09.835438 21767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:08:09.835566 21767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "e5bc667ab99d40fe9f4a6f856d2b4937"
format_stamp: "Formatted at 2026-09-29 03:08:09 on dist-test-slave-bdps"
server_key: "25026daffd09a902877af35d5b2b79a2"
server_key_iv: "d8673b20d45817dc6194c79945384c11"
server_key_version: "encryptionkey@0"
I20260929 03:08:09.835680 21767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:09.854013 21767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:09.857026 21767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:09.857211 21767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:09.864741 21767 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:42781
I20260929 03:08:09.865008 21834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:42781 every 8 connection(s)
I20260929 03:08:09.865839 21767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:08:09.869109 21835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:09.871889 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 21767
I20260929 03:08:09.872035 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:08:09.872393 8005 external_mini_cluster.cc:1428] Setting key 0f284785d7238328ad50d97771015388
I20260929 03:08:09.876248 21835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Bootstrap starting.
I20260929 03:08:09.879230 21835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:09.880127 21835 log.cc:824] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Log is configured to *not* fsync() on all Append() calls
Sep 29 03:08:09 dist-test-slave-bdps krb5kdc[21754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:09.882458 21835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: No bootstrap required, opened a new log
I20260929 03:08:09.886415 21835 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } }
I20260929 03:08:09.886643 21835 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:09.886705 21835 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5bc667ab99d40fe9f4a6f856d2b4937, State: Initialized, Role: FOLLOWER
I20260929 03:08:09.887156 21835 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } }
I20260929 03:08:09.887271 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:09.874063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49634 (local address 127.7.209.126:42781)
0929 03:08:09.874505 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:09.874522 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:09.874563 (+ 41us) server_negotiation.cc:407] Connection header received
0929 03:08:09.875278 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:09.875299 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:09.875685 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:09.876006 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:09.876921 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:09.878027 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:09.879176 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:09.879710 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:09.879833 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:09.882657 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:09.882663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:09.882677 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:09.882704 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:09.884357 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:09.884808 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:09.884812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:09.884816 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:09.884933 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:09.885173 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:09.885176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:09.885177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:09.885571 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:09.885795 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:09.886029 (+ 234us) server_negotiation.cc:299] Negotiation successful
0929 03:08:09.886336 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":123,"threads_started":1}
I20260929 03:08:09.887281 21835 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:08:09.887383 21835 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:08:09.887472 21835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:09.888339 21835 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } }
I20260929 03:08:09.888677 21835 leader_election.cc:302] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5bc667ab99d40fe9f4a6f856d2b4937; no voters:
I20260929 03:08:09.888985 21835 leader_election.cc:288] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:08:09.889062 21840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:08:09.889256 21840 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 1 LEADER]: Becoming Leader. State: Replica: e5bc667ab99d40fe9f4a6f856d2b4937, State: Running, Role: LEADER
I20260929 03:08:09.889653 21840 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } }
I20260929 03:08:09.890373 21835 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:08:09.891675 21842 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5bc667ab99d40fe9f4a6f856d2b4937. Latest consensus state: current_term: 1 leader_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } } }
I20260929 03:08:09.891820 21842 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:09.892009 21841 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } } }
I20260929 03:08:09.892179 21841 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: This master's current role is: LEADER
W20260929 03:08:09.895540 21855 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:08:09.895615 21855 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:08:09.895696 21853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:08:09.896560 21853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:08:09.902292 21853 catalog_manager.cc:1347] Generated new cluster ID: 624a79f3d72e482f94d374dd2f2e9bad
I20260929 03:08:09.902343 21853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:08:09.915515 21853 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:08:09.916555 21853 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:08:09.921891 21853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Generated new TSK 0
I20260929 03:08:09.922609 21853 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:08:09.958096 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:42781
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:10.114869 21861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:10.115193 21861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:10.115249 21861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:10.119405 21861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:10.119463 21861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:10.119525 21861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:08:10.124782 21861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:42781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:10.125982 21861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:10.127162 21861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:10.137781 21867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:10.137797 21866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:10.137786 21869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:10.137938 21861 server_base.cc:1034] running on GCE node
I20260929 03:08:10.138463 21861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:10.139120 21861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:10.140282 21861 hybrid_clock.cc:648] HybridClock initialized: now 1790651290140257 us; error 39 us; skew 500 ppm
Sep 29 03:08:10 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651290, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:10.143149 21861 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:08:10.144224 21861 webserver.cc:460] Webserver started at http://127.7.209.65:45133/ using document root <none> and password file <none>
I20260929 03:08:10.144964 21861 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:10.145045 21861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:10.145284 21861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:08:10.149151 21861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "b892a39fdc7b44b7a35a9a6418e62fcf"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "001814dcaa464b0591a912f9a966cba6"
server_key_iv: "5ff64360c78525aaea871976ccbf5fe0"
server_key_version: "encryptionkey@0"
I20260929 03:08:10.150209 21861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "b892a39fdc7b44b7a35a9a6418e62fcf"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "001814dcaa464b0591a912f9a966cba6"
server_key_iv: "5ff64360c78525aaea871976ccbf5fe0"
server_key_version: "encryptionkey@0"
I20260929 03:08:10.155877 21861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 03:08:10.158851 21876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:10.160228 21861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:08:10.160348 21861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "b892a39fdc7b44b7a35a9a6418e62fcf"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "001814dcaa464b0591a912f9a966cba6"
server_key_iv: "5ff64360c78525aaea871976ccbf5fe0"
server_key_version: "encryptionkey@0"
I20260929 03:08:10.160452 21861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:10.172439 21861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:10.175609 21861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:10.175786 21861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:10.176380 21861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:10.177225 21861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:10.177289 21861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:10.177342 21861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:10.177367 21861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:10.186748 21861 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:32799
I20260929 03:08:10.186774 21989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:32799 every 8 connection(s)
I20260929 03:08:10.187700 21861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:08:10.194474 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 21861
I20260929 03:08:10.194609 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:08:10.194903 8005 external_mini_cluster.cc:1428] Setting key 2a323ef6806c612fbb8338d3834ce18c
Sep 29 03:08:10 dist-test-slave-bdps krb5kdc[21754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651290, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:10.202066 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.189700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:52095 (local address 127.7.209.126:42781)
0929 03:08:10.189849 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:10.189853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:10.190775 (+ 922us) server_negotiation.cc:407] Connection header received
0929 03:08:10.192267 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:10.192271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:10.192336 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:10.192414 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:10.193722 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.194314 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.195573 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.195877 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.195981 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.198562 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:10.198566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:10.198570 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:10.198603 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:10.199978 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:10.200476 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:10.200479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:10.200481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:10.200579 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:10.200881 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:10.200884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:10.200885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:10.201114 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:10.201209 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:10.201701 (+ 492us) server_negotiation.cc:299] Negotiation successful
0929 03:08:10.201873 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 03:08:10.202860 21992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.190135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:52095)
0929 03:08:10.190614 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:10.190662 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:10.192043 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:10.192614 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:10.192622 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:10.193027 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:10.193566 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.193581 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.194434 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.194439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.195457 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.195466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.196019 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.196022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.196204 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.196811 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:10.196827 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:10.198382 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:10.200077 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:10.200081 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:10.200094 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:10.200363 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:10.200657 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:10.200659 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:10.200660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:10.200798 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:10.201195 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:10.201200 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:10.201437 (+ 237us) client_negotiation.cc:769] Sending connection context
0929 03:08:10.201584 (+ 147us) client_negotiation.cc:240] Negotiation successful
0929 03:08:10.201940 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":119,"threads_started":1}
I20260929 03:08:10.204097 21990 heartbeater.cc:347] Connected to a master server at 127.7.209.126:42781
I20260929 03:08:10.204346 21990 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:10.204855 21990 heartbeater.cc:511] Master 127.7.209.126:42781 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:08:10.206489 21799 ts_manager.cc:194] Registered new tserver with Master: b892a39fdc7b44b7a35a9a6418e62fcf (127.7.209.65:32799)
I20260929 03:08:10.207733 21799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:52095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:08:10.221735 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:42781
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:10.345335 21995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:10.345629 21995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:10.345690 21995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:10.349750 21995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:10.349818 21995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:10.349891 21995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:08:10.355198 21995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:42781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:10.356474 21995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:10.357834 21995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:10.368520 22000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:10.368511 22001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:10.368633 21995 server_base.cc:1034] running on GCE node
W20260929 03:08:10.368525 22003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:10.369181 21995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:10.369882 21995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:10.371029 21995 hybrid_clock.cc:648] HybridClock initialized: now 1790651290371014 us; error 28 us; skew 500 ppm
Sep 29 03:08:10 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651290, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:10.373860 21995 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:08:10.374748 21995 webserver.cc:460] Webserver started at http://127.7.209.66:46577/ using document root <none> and password file <none>
I20260929 03:08:10.375339 21995 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:10.375406 21995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:10.375623 21995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:08:10.377931 21995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "97c54f00b42b48ffacfd310fb82c37fe"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "ac5d8dbf82c8bfd59263842af45d265c"
server_key_iv: "25753654f215534ee97560f8d29eec1c"
server_key_version: "encryptionkey@0"
I20260929 03:08:10.378528 21995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "97c54f00b42b48ffacfd310fb82c37fe"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "ac5d8dbf82c8bfd59263842af45d265c"
server_key_iv: "25753654f215534ee97560f8d29eec1c"
server_key_version: "encryptionkey@0"
I20260929 03:08:10.382561 21995 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260929 03:08:10.385118 22010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:10.386499 21995 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 03:08:10.386618 21995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "97c54f00b42b48ffacfd310fb82c37fe"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "ac5d8dbf82c8bfd59263842af45d265c"
server_key_iv: "25753654f215534ee97560f8d29eec1c"
server_key_version: "encryptionkey@0"
I20260929 03:08:10.386719 21995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:10.400247 21995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:10.403200 21995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:10.403386 21995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:10.404023 21995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:10.404932 21995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:10.404980 21995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:10.405036 21995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:10.405066 21995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:10.414536 21995 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:44955
I20260929 03:08:10.414559 22123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:44955 every 8 connection(s)
I20260929 03:08:10.415563 21995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:08:10.417862 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 21995
I20260929 03:08:10.417995 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:08:10.418355 8005 external_mini_cluster.cc:1428] Setting key 8677a795a8e295ffb849ae00de770c76
Sep 29 03:08:10 dist-test-slave-bdps krb5kdc[21754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651290, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:08:10.431926 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.417665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:60083 (local address 127.7.209.126:42781)
0929 03:08:10.417808 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:10.417813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:10.419342 (+ 1529us) server_negotiation.cc:407] Connection header received
0929 03:08:10.420765 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:10.420769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:10.420835 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:10.420918 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:10.422335 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.423066 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.424638 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.424966 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.425071 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.427899 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:10.427903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:10.427905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:10.427934 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:10.429343 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:10.430061 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:10.430064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:10.430066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:10.430164 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:10.430551 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:10.430553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:10.430555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:10.430829 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:10.430920 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:10.431565 (+ 645us) server_negotiation.cc:299] Negotiation successful
0929 03:08:10.431741 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 03:08:10.432893 22126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.418286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:60083)
0929 03:08:10.418885 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:10.418949 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:10.420520 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:10.421105 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:10.421113 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:10.421526 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:10.422193 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.422209 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.423215 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.423219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.424518 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.424527 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.425101 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.425104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.425273 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.425936 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:10.425954 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:10.427702 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:10.429462 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:10.429470 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:10.429493 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:10.429912 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:10.430268 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:10.430271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:10.430273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:10.430468 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:10.430936 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:10.430942 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:10.431266 (+ 324us) client_negotiation.cc:769] Sending connection context
0929 03:08:10.431450 (+ 184us) client_negotiation.cc:240] Negotiation successful
0929 03:08:10.431801 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":164,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
I20260929 03:08:10.434264 22124 heartbeater.cc:347] Connected to a master server at 127.7.209.126:42781
I20260929 03:08:10.434554 22124 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:10.435146 22124 heartbeater.cc:511] Master 127.7.209.126:42781 requested a full tablet report, sending...
I20260929 03:08:10.436420 21799 ts_manager.cc:194] Registered new tserver with Master: 97c54f00b42b48ffacfd310fb82c37fe (127.7.209.66:44955)
I20260929 03:08:10.437080 21799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:60083
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:08:10.448583 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:42781
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:10.575030 22129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:10.575331 22129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:10.575378 22129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:10.579761 22129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:10.579826 22129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:10.579897 22129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:08:10.585392 22129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:42781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:10.586738 22129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:10.588111 22129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:10.599877 22135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:10.599975 22129 server_base.cc:1034] running on GCE node
W20260929 03:08:10.599869 22137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:10.599882 22134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:10.600582 22129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:10.601307 22129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:10.602473 22129 hybrid_clock.cc:648] HybridClock initialized: now 1790651290602444 us; error 40 us; skew 500 ppm
Sep 29 03:08:10 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651290, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:10.605398 22129 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:08:10.606370 22129 webserver.cc:460] Webserver started at http://127.7.209.67:37567/ using document root <none> and password file <none>
I20260929 03:08:10.606976 22129 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:10.607038 22129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:10.607232 22129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:08:10.609572 22129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "2293dbf786864e63b1a063998c33637b"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "419789a7f80abc143395002c5f890f18"
server_key_iv: "8e75e06c24098654408c92d7d7ca5f56"
server_key_version: "encryptionkey@0"
I20260929 03:08:10.610203 22129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "2293dbf786864e63b1a063998c33637b"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "419789a7f80abc143395002c5f890f18"
server_key_iv: "8e75e06c24098654408c92d7d7ca5f56"
server_key_version: "encryptionkey@0"
I20260929 03:08:10.614262 22129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:08:10.616933 22144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:10.618355 22129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:10.618465 22129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "2293dbf786864e63b1a063998c33637b"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "419789a7f80abc143395002c5f890f18"
server_key_iv: "8e75e06c24098654408c92d7d7ca5f56"
server_key_version: "encryptionkey@0"
I20260929 03:08:10.618566 22129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:10.638088 22129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:10.641726 22129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:10.641960 22129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:10.642730 22129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:10.643759 22129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:10.643833 22129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:10.643888 22129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:10.643909 22129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:10.653522 22129 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:32851
I20260929 03:08:10.653544 22257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:32851 every 8 connection(s)
I20260929 03:08:10.654575 22129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:08:10.656057 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 22129
I20260929 03:08:10.656189 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:08:10.656508 8005 external_mini_cluster.cc:1428] Setting key 6bbda38dd220963e19bf2a0675a32532
Sep 29 03:08:10 dist-test-slave-bdps krb5kdc[21754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651290, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:10.669061 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.656719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:50443 (local address 127.7.209.126:42781)
0929 03:08:10.656905 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:10.656909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:10.658062 (+ 1153us) server_negotiation.cc:407] Connection header received
0929 03:08:10.659704 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:10.659707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:10.659761 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:10.659832 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:10.661118 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.661757 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.662933 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.663150 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.663219 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.665802 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:10.665806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:10.665808 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:10.665833 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:10.667179 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:10.667632 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:10.667635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:10.667636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:10.667751 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:10.668010 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:10.668012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:10.668014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:10.668199 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:10.668271 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:10.668794 (+ 523us) server_negotiation.cc:299] Negotiation successful
0929 03:08:10.668913 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:08:10.670214 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.657288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:50443)
0929 03:08:10.657905 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:10.657965 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:10.659469 (+ 1504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:10.660011 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:10.660019 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:10.660439 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:10.661001 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.661015 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.661882 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.661885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.662844 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.662852 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.663258 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.663261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.663441 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.663990 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:10.664006 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:10.665631 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:10.667281 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:10.667286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:10.667299 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:10.667550 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:10.667819 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:10.667821 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:10.667822 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:10.667948 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:10.668268 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:10.668273 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:10.668528 (+ 255us) client_negotiation.cc:769] Sending connection context
0929 03:08:10.668697 (+ 169us) client_negotiation.cc:240] Negotiation successful
0929 03:08:10.668999 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":146,"threads_started":1}
I20260929 03:08:10.671422 22258 heartbeater.cc:347] Connected to a master server at 127.7.209.126:42781
I20260929 03:08:10.671670 22258 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:10.672216 22258 heartbeater.cc:511] Master 127.7.209.126:42781 requested a full tablet report, sending...
I20260929 03:08:10.673360 21798 ts_manager.cc:194] Registered new tserver with Master: 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851)
I20260929 03:08:10.674063 21798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:50443
I20260929 03:08:10.682735 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:08:10.693369 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.684982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.7.209.126:42781)
0929 03:08:10.685110 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:10.685115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:10.685312 (+ 197us) server_negotiation.cc:407] Connection header received
0929 03:08:10.685443 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:10.685446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:10.685497 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:10.685562 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:10.686556 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.687244 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.688098 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.688315 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.688382 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.689765 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:10.689769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:10.689772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:10.689797 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:10.691727 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:10.692086 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:10.692088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:10.692090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:10.692186 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:10.692575 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:10.692578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:10.692579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:10.692861 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:10.692979 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:10.693132 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 03:08:10.693236 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 03:08:10.696333 21798 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:08:10.699935 21798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:08:10.715869 22271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.709494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56328 (local address 127.7.209.65:32799)
0929 03:08:10.710421 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:10.710430 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:10.710459 (+ 29us) server_negotiation.cc:407] Connection header received
0929 03:08:10.710555 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:10.710561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:10.710828 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:10.710990 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:10.711274 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.712376 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.714415 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.715108 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.715218 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.715409 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:10.715544 (+ 135us) server_negotiation.cc:299] Negotiation successful
0929 03:08:10.715664 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":170,"threads_started":1}
I20260929 03:08:10.715943 22269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.709397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:32851 (local address 127.0.0.1:36342)
0929 03:08:10.710349 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:10.710361 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:10.710436 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:10.711659 (+ 1223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:10.711663 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:10.711681 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:10.711849 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.711856 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.713166 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.713169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.714269 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.714278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.715381 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.715385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.715485 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.715508 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 03:08:10.715554 (+ 46us) client_negotiation.cc:240] Negotiation successful
0929 03:08:10.715774 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"mutex_wait_us":132,"spinlock_wait_cycles":3456,"thread_start_us":105,"threads_started":1}
I20260929 03:08:10.716173 22273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.710204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:44955 (local address 127.0.0.1:59662)
0929 03:08:10.710767 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:10.710780 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:10.710882 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:10.711682 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:10.711685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:10.711704 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:10.711972 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.711980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.713200 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.713203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.714466 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.714473 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.715798 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.715802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.715882 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.715904 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 03:08:10.715961 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 03:08:10.716032 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":192,"thread_start_us":113,"threads_started":1}
I20260929 03:08:10.716408 22272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.709865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.7.209.67:32851)
0929 03:08:10.711149 (+ 1284us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:10.711162 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:10.711187 (+ 25us) server_negotiation.cc:407] Connection header received
0929 03:08:10.711274 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:10.711288 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:10.711500 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:10.711700 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:10.711993 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.713011 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.714388 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.715233 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.715674 (+ 441us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.715843 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:10.715898 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 03:08:10.716249 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1167,"thread_start_us":191,"threads_started":1}
I20260929 03:08:10.716701 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.710753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59662 (local address 127.7.209.66:44955)
0929 03:08:10.711261 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:10.711273 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:10.711301 (+ 28us) server_negotiation.cc:407] Connection header received
0929 03:08:10.711377 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:10.711382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:10.711525 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:10.711791 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:10.712087 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.713043 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.714582 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.715377 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.715470 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.716375 (+ 905us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:10.716432 (+ 57us) server_negotiation.cc:299] Negotiation successful
0929 03:08:10.716542 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":139,"threads_started":1}
I20260929 03:08:10.717801 22270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.709301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:32799 (local address 127.0.0.1:56328)
0929 03:08:10.710173 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:10.710235 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:10.710393 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:10.710985 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:10.710993 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:10.711018 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:10.711166 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.711173 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.712499 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.712502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.714278 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.714288 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.715231 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.715235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.715343 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.715381 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 03:08:10.715503 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 03:08:10.717684 (+ 2181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":57,"thread_start_us":108,"threads_started":1}
I20260929 03:08:10.720166 22192 tablet_service.cc:1467] Processing CreateTablet for tablet 440306346e764dd7adf72f4d4a040a89 (DEFAULT_TABLE table=test-table [id=58c36b5f720f437a861e07db9594fec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:10.720158 22058 tablet_service.cc:1467] Processing CreateTablet for tablet 440306346e764dd7adf72f4d4a040a89 (DEFAULT_TABLE table=test-table [id=58c36b5f720f437a861e07db9594fec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:10.720605 21924 tablet_service.cc:1467] Processing CreateTablet for tablet 440306346e764dd7adf72f4d4a040a89 (DEFAULT_TABLE table=test-table [id=58c36b5f720f437a861e07db9594fec3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:10.721395 22058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 440306346e764dd7adf72f4d4a040a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:10.721415 22192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 440306346e764dd7adf72f4d4a040a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:10.721481 21924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 440306346e764dd7adf72f4d4a040a89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:10.728194 22275 tablet_bootstrap.cc:492] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe: Bootstrap starting.
I20260929 03:08:10.728878 22277 tablet_bootstrap.cc:492] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Bootstrap starting.
I20260929 03:08:10.729902 22276 tablet_bootstrap.cc:492] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b: Bootstrap starting.
I20260929 03:08:10.731441 22277 tablet_bootstrap.cc:654] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:10.731866 22276 tablet_bootstrap.cc:654] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:10.731870 22275 tablet_bootstrap.cc:654] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:10.732424 22277 log.cc:824] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:10.732679 22275 log.cc:824] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:10.732854 22276 log.cc:824] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:10.734565 22277 tablet_bootstrap.cc:492] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: No bootstrap required, opened a new log
I20260929 03:08:10.734609 22276 tablet_bootstrap.cc:492] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b: No bootstrap required, opened a new log
I20260929 03:08:10.734609 22275 tablet_bootstrap.cc:492] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe: No bootstrap required, opened a new log
I20260929 03:08:10.734778 22277 ts_tablet_manager.cc:1397] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 03:08:10.734778 22276 ts_tablet_manager.cc:1397] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 03:08:10.734778 22275 ts_tablet_manager.cc:1397] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260929 03:08:10.738400 22277 raft_consensus.cc:348] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } } peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } }
I20260929 03:08:10.738617 22277 raft_consensus.cc:374] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:10.738672 22277 raft_consensus.cc:729] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b892a39fdc7b44b7a35a9a6418e62fcf, State: Initialized, Role: FOLLOWER
I20260929 03:08:10.738559 22275 raft_consensus.cc:348] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } } peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } }
I20260929 03:08:10.738827 22275 raft_consensus.cc:374] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:10.738895 22275 raft_consensus.cc:729] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c54f00b42b48ffacfd310fb82c37fe, State: Initialized, Role: FOLLOWER
I20260929 03:08:10.738850 22276 raft_consensus.cc:348] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } } peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } }
I20260929 03:08:10.739107 22276 raft_consensus.cc:374] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:10.739163 22276 raft_consensus.cc:729] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2293dbf786864e63b1a063998c33637b, State: Initialized, Role: FOLLOWER
I20260929 03:08:10.739179 22277 consensus_queue.cc:260] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } } peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } }
I20260929 03:08:10.739341 22275 consensus_queue.cc:260] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } } peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } }
I20260929 03:08:10.739673 22276 consensus_queue.cc:260] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } } peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } }
I20260929 03:08:10.739984 21990 heartbeater.cc:503] Master 127.7.209.126:42781 was elected leader, sending a full tablet report...
I20260929 03:08:10.740334 22275 ts_tablet_manager.cc:1428] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 03:08:10.740528 22276 ts_tablet_manager.cc:1428] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260929 03:08:10.740658 22124 heartbeater.cc:503] Master 127.7.209.126:42781 was elected leader, sending a full tablet report...
I20260929 03:08:10.741081 22258 heartbeater.cc:503] Master 127.7.209.126:42781 was elected leader, sending a full tablet report...
I20260929 03:08:10.742264 22277 ts_tablet_manager.cc:1428] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260929 03:08:10.803823 22282 raft_consensus.cc:482] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:08:10.804092 22282 raft_consensus.cc:504] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } } peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } }
I20260929 03:08:10.805514 22282 leader_election.cc:288] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b892a39fdc7b44b7a35a9a6418e62fcf (127.7.209.65:32799), 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851)
I20260929 03:08:10.810853 22271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.806214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:54179 (local address 127.7.209.65:32799)
0929 03:08:10.806350 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:10.806354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:10.806367 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:08:10.806424 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:10.806428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:10.806487 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:10.806579 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:10.806878 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.807764 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.809538 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.810205 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.810334 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.810395 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:10.810629 (+ 234us) server_negotiation.cc:299] Negotiation successful
0929 03:08:10.810703 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:08:10.810853 22272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.805930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:54759 (local address 127.7.209.67:32851)
0929 03:08:10.806082 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:10.806086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:10.806457 (+ 371us) server_negotiation.cc:407] Connection header received
0929 03:08:10.806545 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:10.806548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:10.806604 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:10.806672 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:10.806922 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.807738 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.809538 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.810205 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.810333 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.810395 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:10.810629 (+ 234us) server_negotiation.cc:299] Negotiation successful
0929 03:08:10.810700 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 03:08:10.810864 22126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.805802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:32799 (local address 127.7.209.66:54179)
0929 03:08:10.805919 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:10.805936 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:10.806039 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:10.806587 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:10.806590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:10.806603 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:10.806754 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.806762 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.807854 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.807856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.809412 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.809424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.810332 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.810337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.810493 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.810545 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 03:08:10.810614 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 03:08:10.810733 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":180}
I20260929 03:08:10.810873 22284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.805802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:32851 (local address 127.7.209.66:54759)
0929 03:08:10.806354 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:10.806365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:10.806463 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:10.806708 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:10.806710 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:10.806725 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:10.806825 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.806831 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.807843 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.807847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.809412 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:10.809424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.810332 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.810337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:10.810489 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.810534 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 03:08:10.810614 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 03:08:10.810708 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":251,"thread_start_us":172,"threads_started":1}
I20260929 03:08:10.811861 21944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "440306346e764dd7adf72f4d4a040a89" candidate_uuid: "97c54f00b42b48ffacfd310fb82c37fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" is_pre_election: true
I20260929 03:08:10.811898 22212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "440306346e764dd7adf72f4d4a040a89" candidate_uuid: "97c54f00b42b48ffacfd310fb82c37fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2293dbf786864e63b1a063998c33637b" is_pre_election: true
I20260929 03:08:10.812274 21944 raft_consensus.cc:2454] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c54f00b42b48ffacfd310fb82c37fe in term 0.
I20260929 03:08:10.812274 22212 raft_consensus.cc:2454] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c54f00b42b48ffacfd310fb82c37fe in term 0.
I20260929 03:08:10.812837 22012 leader_election.cc:302] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c54f00b42b48ffacfd310fb82c37fe, b892a39fdc7b44b7a35a9a6418e62fcf; no voters:
I20260929 03:08:10.813153 22282 raft_consensus.cc:2781] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:08:10.813237 22282 raft_consensus.cc:482] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:08:10.813294 22282 raft_consensus.cc:3037] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:10.814538 22282 raft_consensus.cc:504] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } } peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } }
I20260929 03:08:10.814889 22282 leader_election.cc:288] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [CANDIDATE]: Term 1 election: Requested vote from peers b892a39fdc7b44b7a35a9a6418e62fcf (127.7.209.65:32799), 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851)
I20260929 03:08:10.815222 21944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "440306346e764dd7adf72f4d4a040a89" candidate_uuid: "97c54f00b42b48ffacfd310fb82c37fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf"
I20260929 03:08:10.815310 22212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "440306346e764dd7adf72f4d4a040a89" candidate_uuid: "97c54f00b42b48ffacfd310fb82c37fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2293dbf786864e63b1a063998c33637b"
I20260929 03:08:10.815366 21944 raft_consensus.cc:3037] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:10.815430 22212 raft_consensus.cc:3037] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:10.816844 21944 raft_consensus.cc:2454] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c54f00b42b48ffacfd310fb82c37fe in term 1.
I20260929 03:08:10.816844 22212 raft_consensus.cc:2454] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c54f00b42b48ffacfd310fb82c37fe in term 1.
I20260929 03:08:10.817118 22012 leader_election.cc:302] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97c54f00b42b48ffacfd310fb82c37fe, b892a39fdc7b44b7a35a9a6418e62fcf; no voters:
I20260929 03:08:10.817320 22282 raft_consensus.cc:2781] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:08:10.817543 22282 raft_consensus.cc:686] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 LEADER]: Becoming Leader. State: Replica: 97c54f00b42b48ffacfd310fb82c37fe, State: Running, Role: LEADER
I20260929 03:08:10.817970 22282 consensus_queue.cc:237] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } } peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } }
I20260929 03:08:10.821561 21798 catalog_manager.cc:5495] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 97c54f00b42b48ffacfd310fb82c37fe (127.7.209.66). New cstate: current_term: 1 leader_uuid: "97c54f00b42b48ffacfd310fb82c37fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } health_report { overall_health: HEALTHY } } }
I20260929 03:08:10.836462 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.832737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.7.209.66:44955)
0929 03:08:10.832880 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:10.832884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:10.832898 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:08:10.833000 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:10.833002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:10.833071 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:10.833170 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:08:10.833383 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.834062 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.834953 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:10.835161 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:10.835228 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:10.835526 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:08:10.836084 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:08:10.836184 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:10.836210 (+ 26us) server_negotiation.cc:299] Negotiation successful
0929 03:08:10.836285 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:08:10.844100 21944 raft_consensus.cc:1261] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Refusing update from remote peer 97c54f00b42b48ffacfd310fb82c37fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:08:10.844100 22212 raft_consensus.cc:1261] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b [term 1 FOLLOWER]: Refusing update from remote peer 97c54f00b42b48ffacfd310fb82c37fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:08:10.844923 22285 consensus_queue.cc:1035] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:08:10.845124 22282 consensus_queue.cc:1035] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:08:10.852520 22288 mvcc.cc:205] Tried to move back new op lower bound from 7334507687288840192 to 7334507687193378816. Current Snapshot: MvccSnapshot[applied={T|T < 7334507687288840192}]
I20260929 03:08:10.852830 22289 mvcc.cc:205] Tried to move back new op lower bound from 7334507687288840192 to 7334507687193378816. Current Snapshot: MvccSnapshot[applied={T|T < 7334507687288840192}]
I20260929 03:08:10.853899 22290 mvcc.cc:205] Tried to move back new op lower bound from 7334507687288840192 to 7334507687193378816. Current Snapshot: MvccSnapshot[applied={T|T < 7334507687288840192}]
I20260929 03:08:10.861761 21797 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:08:10.861928 21797 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:08:10.864446 21797 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Sending DeleteTablet for 3 replicas of tablet 440306346e764dd7adf72f4d4a040a89
I20260929 03:08:10.865415 22192 tablet_service.cc:1514] Processing DeleteTablet for tablet 440306346e764dd7adf72f4d4a040a89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:10 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:36342
I20260929 03:08:10.865520 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 21861
I20260929 03:08:10.866065 22058 tablet_service.cc:1514] Processing DeleteTablet for tablet 440306346e764dd7adf72f4d4a040a89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:10 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:59662
I20260929 03:08:10.869894 22297 tablet_replica.cc:330] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b: stopping tablet replica
I20260929 03:08:10.870325 22297 raft_consensus.cc:2229] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:08:10.870744 22297 raft_consensus.cc:2258] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:10.871179 22298 tablet_replica.cc:330] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe: stopping tablet replica
I20260929 03:08:10.871521 22298 raft_consensus.cc:2229] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:08:10.871938 22298 raft_consensus.cc:2258] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:10.873929 22298 ts_tablet_manager.cc:1905] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:08:10.876263 22297 ts_tablet_manager.cc:1905] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 03:08:10.876555 21784 connection.cc:535] client connection to 127.7.209.65:32799 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260929 03:08:10.876719 21784 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 03:08:10.877235 22298 ts_tablet_manager.cc:1918] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:08:10.877345 22298 log.cc:1196] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/440306346e764dd7adf72f4d4a040a89
I20260929 03:08:10.877765 22298 ts_tablet_manager.cc:1939] T 440306346e764dd7adf72f4d4a040a89 P 97c54f00b42b48ffacfd310fb82c37fe: Deleting consensus metadata
I20260929 03:08:10.879217 21786 catalog_manager.cc:4841] TS 97c54f00b42b48ffacfd310fb82c37fe (127.7.209.66:44955): tablet 440306346e764dd7adf72f4d4a040a89 (table test-table [id=58c36b5f720f437a861e07db9594fec3]) successfully deleted
I20260929 03:08:10.879590 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 21995
I20260929 03:08:10.882046 22297 ts_tablet_manager.cc:1918] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:08:10.882143 22297 log.cc:1196] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/440306346e764dd7adf72f4d4a040a89
I20260929 03:08:10.882442 22297 ts_tablet_manager.cc:1939] T 440306346e764dd7adf72f4d4a040a89 P 2293dbf786864e63b1a063998c33637b: Deleting consensus metadata
I20260929 03:08:10.883705 21786 catalog_manager.cc:4841] TS 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851): tablet 440306346e764dd7adf72f4d4a040a89 (table test-table [id=58c36b5f720f437a861e07db9594fec3]) successfully deleted
I20260929 03:08:10.884387 22270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:10.884059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:32799 (local address 127.0.0.1:56334)
0929 03:08:10.884214 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:10.884297 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:32799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 03:08:10.884550 21784 catalog_manager.cc:4584] TS b892a39fdc7b44b7a35a9a6418e62fcf (127.7.209.65:32799): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 440306346e764dd7adf72f4d4a040a89: Network error: Client connection negotiation failed: client connection to 127.7.209.65:32799: connect: Connection refused (error 111)
I20260929 03:08:10.887219 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 22129
I20260929 03:08:10.892788 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 21767
I20260929 03:08:10.899072 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:42781
--webserver_interface=127.7.209.126
--webserver_port=46451
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:42781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:11.023160 22300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:11.023439 22300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:11.023476 22300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:11.027379 22300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:08:11.027424 22300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:11.027441 22300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:11.027456 22300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:08:11.027472 22300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:08:11.033110 22300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:42781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:42781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=46451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:11.034333 22300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:11.035475 22300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:11.050035 22307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:11.050094 22300 server_base.cc:1034] running on GCE node
W20260929 03:08:11.050028 22306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:11.050028 22309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:11.050956 22300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:11.052217 22300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:11.053380 22300 hybrid_clock.cc:648] HybridClock initialized: now 1790651291053355 us; error 37 us; skew 500 ppm
Sep 29 03:08:11 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651291, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:11.056290 22300 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:08:11.057271 22300 webserver.cc:460] Webserver started at http://127.7.209.126:46451/ using document root <none> and password file <none>
I20260929 03:08:11.057966 22300 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:11.058048 22300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:11.062458 22300 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:08:11.064802 22316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:11.066184 22300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 03:08:11.066303 22300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "e5bc667ab99d40fe9f4a6f856d2b4937"
format_stamp: "Formatted at 2026-09-29 03:08:09 on dist-test-slave-bdps"
server_key: "25026daffd09a902877af35d5b2b79a2"
server_key_iv: "d8673b20d45817dc6194c79945384c11"
server_key_version: "encryptionkey@0"
I20260929 03:08:11.066996 22300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:11.076808 22300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:11.079892 22300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:11.080097 22300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:11.088222 22300 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:42781
I20260929 03:08:11.088512 22368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:42781 every 8 connection(s)
I20260929 03:08:11.089654 22300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:08:11.095522 22369 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:08:11.096145 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 22300
I20260929 03:08:11.096637 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:32799
--local_ip_for_outbound_sockets=127.7.209.65
--tserver_master_addrs=127.7.209.126:42781
--webserver_port=45133
--webserver_interface=127.7.209.65
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:08:11.097851 22369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Bootstrap starting.
I20260929 03:08:11.118141 22369 log.cc:824] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:11.128322 22369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:08:11.128794 22369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Bootstrap complete.
I20260929 03:08:11.134801 22369 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } }
I20260929 03:08:11.135357 22369 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5bc667ab99d40fe9f4a6f856d2b4937, State: Initialized, Role: FOLLOWER
I20260929 03:08:11.136076 22369 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } }
I20260929 03:08:11.136250 22369 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:08:11.136329 22369 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:08:11.136432 22369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 1 FOLLOWER]: Advancing to term 2
I20260929 03:08:11.138530 22369 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } }
I20260929 03:08:11.138870 22369 leader_election.cc:302] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5bc667ab99d40fe9f4a6f856d2b4937; no voters:
I20260929 03:08:11.139163 22369 leader_election.cc:288] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 03:08:11.139303 22373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 03:08:11.139564 22373 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 2 LEADER]: Becoming Leader. State: Replica: e5bc667ab99d40fe9f4a6f856d2b4937, State: Running, Role: LEADER
I20260929 03:08:11.139897 22373 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } }
I20260929 03:08:11.140398 22369 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:08:11.141364 22375 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5bc667ab99d40fe9f4a6f856d2b4937. Latest consensus state: current_term: 2 leader_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } } }
I20260929 03:08:11.141497 22375 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:11.141755 22374 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } } }
I20260929 03:08:11.141831 22374 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:11.146142 22388 catalog_manager.cc:1259] Loaded cluster ID: 624a79f3d72e482f94d374dd2f2e9bad
I20260929 03:08:11.146198 22388 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: loading cluster ID for follower catalog manager: success
W20260929 03:08:11.147233 22388 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260929 03:08:11.147315 22388 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260929 03:08:11.147388 22382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:08:11.150118 22382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:08:11.150590 22382 catalog_manager.cc:1259] Loaded cluster ID: 624a79f3d72e482f94d374dd2f2e9bad
I20260929 03:08:11.150648 22382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:08:11.151680 22382 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 03:08:11.155956 22382 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 03:08:11.162204 22382 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:08:11.163120 22382 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Loaded TSK: 0
I20260929 03:08:11.164532 22382 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 03:08:11.232563 22371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:11.232903 22371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:11.232952 22371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:11.237150 22371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:11.237224 22371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:11.237293 22371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:08:11.243029 22371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:32799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=45133
--tserver_master_addrs=127.7.209.126:42781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:11.244252 22371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:11.245434 22371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:11.255957 22397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:11.256074 22371 server_base.cc:1034] running on GCE node
W20260929 03:08:11.255957 22396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:11.255949 22399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:11.256620 22371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:11.257324 22371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:11.258482 22371 hybrid_clock.cc:648] HybridClock initialized: now 1790651291258445 us; error 61 us; skew 500 ppm
Sep 29 03:08:11 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651291, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:11.261338 22371 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:08:11.262352 22371 webserver.cc:460] Webserver started at http://127.7.209.65:45133/ using document root <none> and password file <none>
I20260929 03:08:11.262954 22371 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:11.263033 22371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:11.267395 22371 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260929 03:08:11.269881 22406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:11.271457 22371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:08:11.271576 22371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "b892a39fdc7b44b7a35a9a6418e62fcf"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "001814dcaa464b0591a912f9a966cba6"
server_key_iv: "5ff64360c78525aaea871976ccbf5fe0"
server_key_version: "encryptionkey@0"
I20260929 03:08:11.272236 22371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:11.292233 22371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:11.296036 22371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:11.296226 22371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:11.296909 22371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:11.298125 22413 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 03:08:11.302875 22371 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 03:08:11.302930 22371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260929 03:08:11.302986 22371 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 03:08:11.304836 22371 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 03:08:11.304880 22371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260929 03:08:11.305197 22413 tablet_bootstrap.cc:492] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Bootstrap starting.
I20260929 03:08:11.315822 22371 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:32799
I20260929 03:08:11.315923 22520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:32799 every 8 connection(s)
I20260929 03:08:11.316939 22371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:08:11.325474 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 22371
I20260929 03:08:11.326076 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:44955
--local_ip_for_outbound_sockets=127.7.209.66
--tserver_master_addrs=127.7.209.126:42781
--webserver_port=46577
--webserver_interface=127.7.209.66
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:08:11.326131 22413 log.cc:824] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Log is configured to *not* fsync() on all Append() calls
Sep 29 03:08:11 dist-test-slave-bdps krb5kdc[21754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651291, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:11.334973 22413 tablet_bootstrap.cc:492] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:08:11.335320 22413 tablet_bootstrap.cc:492] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Bootstrap complete.
I20260929 03:08:11.336526 22413 ts_tablet_manager.cc:1397] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Time spent bootstrapping tablet: real 0.032s user 0.021s sys 0.008s
I20260929 03:08:11.337738 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.319523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:36997)
0929 03:08:11.320257 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:11.320337 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:11.321596 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:11.322893 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:11.322903 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:11.323516 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:11.324274 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:11.324295 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.325702 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.325706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:11.326869 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:11.326880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.327886 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.327890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:11.328066 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.328799 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:11.328820 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:11.330779 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:11.334052 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:11.334058 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:11.334074 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:11.334436 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:11.334850 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:11.334853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:11.334854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:11.335049 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:11.335847 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:11.335853 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:11.336130 (+ 277us) client_negotiation.cc:769] Sending connection context
0929 03:08:11.336318 (+ 188us) client_negotiation.cc:240] Negotiation successful
0929 03:08:11.336699 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":170,"threads_started":1}
I20260929 03:08:11.338232 22524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.319667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:36997 (local address 127.7.209.126:42781)
0929 03:08:11.320233 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:11.320255 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:11.320496 (+ 241us) server_negotiation.cc:407] Connection header received
0929 03:08:11.322129 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:11.322141 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:11.322464 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:11.322786 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:11.324437 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.325495 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.327006 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.327731 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.327906 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.330992 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:11.330999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:11.331014 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:11.331045 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:11.333925 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:11.334574 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:11.334581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:11.334587 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:11.334748 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:11.335175 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:11.335179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:11.335181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:11.335710 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:11.335858 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:11.336918 (+ 1060us) server_negotiation.cc:299] Negotiation successful
0929 03:08:11.337272 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":139,"threads_started":1}
I20260929 03:08:11.339476 22413 raft_consensus.cc:348] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } } peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } }
I20260929 03:08:11.339982 22413 raft_consensus.cc:729] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b892a39fdc7b44b7a35a9a6418e62fcf, State: Initialized, Role: FOLLOWER
I20260929 03:08:11.340463 22413 consensus_queue.cc:260] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } } peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } }
I20260929 03:08:11.341367 22521 heartbeater.cc:347] Connected to a master server at 127.7.209.126:42781
I20260929 03:08:11.341487 22413 ts_tablet_manager.cc:1428] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260929 03:08:11.341646 22521 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:11.342258 22521 heartbeater.cc:511] Master 127.7.209.126:42781 requested a full tablet report, sending...
I20260929 03:08:11.344964 22333 ts_manager.cc:194] Registered new tserver with Master: b892a39fdc7b44b7a35a9a6418e62fcf (127.7.209.65:32799)
I20260929 03:08:11.348634 22333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:36997
I20260929 03:08:11.351333 22521 heartbeater.cc:503] Master 127.7.209.126:42781 was elected leader, sending a full tablet report...
Sep 29 03:08:11 dist-test-slave-bdps krb5kdc[21754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651291, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
I20260929 03:08:11.359248 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.347993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:32799 (local address 127.0.0.1:56336)
0929 03:08:11.348752 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:11.348803 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:11.348953 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:11.349488 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:11.349495 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:11.349946 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:11.350082 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:11.350089 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.351455 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.351458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:11.352337 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:11.352348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.353103 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.353107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:11.353215 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.353867 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:11.353879 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:11.355492 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:11.357577 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:11.357585 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:11.357590 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:11.357866 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:11.358164 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:11.358167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:11.358168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:11.358250 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:11.358736 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:11.358741 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:11.358841 (+ 100us) client_negotiation.cc:769] Sending connection context
0929 03:08:11.358945 (+ 104us) client_negotiation.cc:240] Negotiation successful
0929 03:08:11.359102 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":103,"threads_started":1}
I20260929 03:08:11.359409 22529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.348086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56336 (local address 127.7.209.65:32799)
0929 03:08:11.348453 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:11.348464 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:11.348903 (+ 439us) server_negotiation.cc:407] Connection header received
0929 03:08:11.349110 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:11.349115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:11.349330 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:11.349474 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:11.350209 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.351319 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.352482 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.352978 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.353084 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.355627 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:11.355637 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:11.355658 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:11.355685 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:11.357458 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:11.357966 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:11.357972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:11.357974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:11.358073 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:11.358333 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:11.358336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:11.358338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:11.358629 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:11.358775 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:11.359058 (+ 283us) server_negotiation.cc:299] Negotiation successful
0929 03:08:11.359248 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":101,"threads_started":1}
I20260929 03:08:11.360422 22455 tablet_service.cc:1514] Processing DeleteTablet for tablet 440306346e764dd7adf72f4d4a040a89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:10 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:56336
I20260929 03:08:11.361363 22530 tablet_replica.cc:330] stopping tablet replica
I20260929 03:08:11.361747 22530 raft_consensus.cc:2229] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:08:11.362040 22530 raft_consensus.cc:2258] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:11.363284 22530 ts_tablet_manager.cc:1905] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:08:11.368530 22530 ts_tablet_manager.cc:1918] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:08:11.368642 22530 log.cc:1196] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal/wals/440306346e764dd7adf72f4d4a040a89
I20260929 03:08:11.368950 22530 ts_tablet_manager.cc:1939] T 440306346e764dd7adf72f4d4a040a89 P b892a39fdc7b44b7a35a9a6418e62fcf: Deleting consensus metadata
I20260929 03:08:11.371259 22318 catalog_manager.cc:4841] TS b892a39fdc7b44b7a35a9a6418e62fcf (127.7.209.65:32799): tablet 440306346e764dd7adf72f4d4a040a89 (table test-table [id=58c36b5f720f437a861e07db9594fec3]) successfully deleted
W20260929 03:08:11.463148 22526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:11.463444 22526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:11.463488 22526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:11.467562 22526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:11.467625 22526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:11.467711 22526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:08:11.473259 22526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:44955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=46577
--tserver_master_addrs=127.7.209.126:42781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:11.474643 22526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:11.475925 22526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:11.487072 22535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:11.487079 22536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:11.487092 22538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:11.487425 22526 server_base.cc:1034] running on GCE node
I20260929 03:08:11.487871 22526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:11.488567 22526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:11.489671 22526 hybrid_clock.cc:648] HybridClock initialized: now 1790651291489642 us; error 42 us; skew 500 ppm
Sep 29 03:08:11 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651291, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:11.492569 22526 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:08:11.493505 22526 webserver.cc:460] Webserver started at http://127.7.209.66:46577/ using document root <none> and password file <none>
I20260929 03:08:11.494170 22526 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:11.494252 22526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:11.498626 22526 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:08:11.500970 22545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:11.502573 22526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:08:11.502707 22526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "97c54f00b42b48ffacfd310fb82c37fe"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "ac5d8dbf82c8bfd59263842af45d265c"
server_key_iv: "25753654f215534ee97560f8d29eec1c"
server_key_version: "encryptionkey@0"
I20260929 03:08:11.503321 22526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:11.527760 22526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:11.531188 22526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:11.531406 22526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:11.532140 22526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:11.533119 22526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:11.533170 22526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:11.533233 22526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:11.533258 22526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:11.542956 22526 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:44955
I20260929 03:08:11.542986 22658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:44955 every 8 connection(s)
I20260929 03:08:11.544106 22526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:08:11.544940 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 22526
I20260929 03:08:11.545493 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:32851
--local_ip_for_outbound_sockets=127.7.209.67
--tserver_master_addrs=127.7.209.126:42781
--webserver_port=37567
--webserver_interface=127.7.209.67
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:08:11 dist-test-slave-bdps krb5kdc[21754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651291, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:11.560663 22524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.546413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:39437 (local address 127.7.209.126:42781)
0929 03:08:11.546591 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:11.546596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:11.547690 (+ 1094us) server_negotiation.cc:407] Connection header received
0929 03:08:11.549228 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:11.549231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:11.549290 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:11.549370 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:11.551183 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.551787 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.553564 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.553884 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.553979 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.557143 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:11.557146 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:11.557149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:11.557178 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:11.558577 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:11.559143 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:11.559145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:11.559147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:11.559220 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:11.559564 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:11.559566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:11.559567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:11.559744 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:11.559817 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:11.560386 (+ 569us) server_negotiation.cc:299] Negotiation successful
0929 03:08:11.560507 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 03:08:11.561931 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.546900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:39437)
0929 03:08:11.547495 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:11.547567 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:11.548949 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:11.549629 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:11.549640 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:11.550181 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:11.550993 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:11.551020 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.551921 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.551924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:11.553421 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:11.553432 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.554059 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.554063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:11.554301 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.554989 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:11.555012 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:11.556954 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:11.558683 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:11.558690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:11.558707 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:11.559056 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:11.559300 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:11.559303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:11.559306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:11.559479 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:11.559826 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:11.559833 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:11.560154 (+ 321us) client_negotiation.cc:769] Sending connection context
0929 03:08:11.560368 (+ 214us) client_negotiation.cc:240] Negotiation successful
0929 03:08:11.560725 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":145,"threads_started":1}
I20260929 03:08:11.563210 22659 heartbeater.cc:347] Connected to a master server at 127.7.209.126:42781
I20260929 03:08:11.563498 22659 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:11.564137 22659 heartbeater.cc:511] Master 127.7.209.126:42781 requested a full tablet report, sending...
I20260929 03:08:11.565398 22333 ts_manager.cc:194] Registered new tserver with Master: 97c54f00b42b48ffacfd310fb82c37fe (127.7.209.66:44955)
I20260929 03:08:11.566066 22333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:39437
W20260929 03:08:11.673489 22662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:11.673834 22662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:11.673883 22662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:11.677958 22662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:11.678026 22662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:11.678100 22662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:08:11.683723 22662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:32851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=37567
--tserver_master_addrs=127.7.209.126:42781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:11.685114 22662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:11.686482 22662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:11.697929 22667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:11.697929 22668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:11.698081 22662 server_base.cc:1034] running on GCE node
W20260929 03:08:11.698263 22670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:11.698697 22662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:11.699429 22662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:11.700595 22662 hybrid_clock.cc:648] HybridClock initialized: now 1790651291700573 us; error 34 us; skew 500 ppm
Sep 29 03:08:11 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651291, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:11.703680 22662 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:08:11.704728 22662 webserver.cc:460] Webserver started at http://127.7.209.67:37567/ using document root <none> and password file <none>
I20260929 03:08:11.705391 22662 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:11.705480 22662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:11.710311 22662 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:11.712889 22677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:11.714461 22662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:11.714581 22662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "2293dbf786864e63b1a063998c33637b"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "419789a7f80abc143395002c5f890f18"
server_key_iv: "8e75e06c24098654408c92d7d7ca5f56"
server_key_version: "encryptionkey@0"
I20260929 03:08:11.715233 22662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:11.739032 22662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:11.742709 22662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:11.742918 22662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:11.743546 22662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:11.744477 22662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:11.744526 22662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:11.744582 22662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:11.744611 22662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:11.754712 22662 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:32851
I20260929 03:08:11.754734 22790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:32851 every 8 connection(s)
I20260929 03:08:11.755952 22662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:08:11.764321 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 22662
Sep 29 03:08:11 dist-test-slave-bdps krb5kdc[21754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651291, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:11.771996 22524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.758275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:54299 (local address 127.7.209.126:42781)
0929 03:08:11.758452 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:11.758456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:11.759448 (+ 992us) server_negotiation.cc:407] Connection header received
0929 03:08:11.760893 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:11.760897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:11.760965 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:11.761046 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:11.762569 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.763156 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.764664 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.764961 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.765081 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.767952 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:11.767955 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:11.767958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:11.768024 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:11.769643 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:11.770143 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:11.770147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:11.770149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:11.770241 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:11.770822 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:11.770826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:11.770828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:11.771058 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:11.771153 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:11.771684 (+ 531us) server_negotiation.cc:299] Negotiation successful
0929 03:08:11.771804 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 03:08:11.773216 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.765143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49644 (local address 127.7.209.126:42781)
0929 03:08:11.765427 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:11.765431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:11.765728 (+ 297us) server_negotiation.cc:407] Connection header received
0929 03:08:11.765884 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:11.765887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:11.765939 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:11.766040 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:11.766824 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.767429 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.768342 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.768668 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.768769 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.770153 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:11.770157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:11.770160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:11.770185 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:11.771606 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:11.772022 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:11.772025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:11.772027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:11.772121 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:11.772421 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:11.772425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:11.772427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:11.772619 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:11.772866 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:11.772934 (+ 68us) server_negotiation.cc:299] Negotiation successful
0929 03:08:11.773050 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":100,"threads_started":1}
I20260929 03:08:11.773233 22793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.758773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:54299)
0929 03:08:11.759286 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:11.759333 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:11.760655 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:11.761262 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:11.761270 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:11.761792 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:11.762414 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:11.762431 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.763361 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.763364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:11.764539 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:11.764552 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.765129 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.765133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:11.765356 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.766056 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:11.766077 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:11.767787 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:11.769757 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:11.769763 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:11.769781 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:11.770050 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:11.770329 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:11.770332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:11.770334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:11.770729 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:11.771150 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:11.771158 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:11.771433 (+ 275us) client_negotiation.cc:769] Sending connection context
0929 03:08:11.771635 (+ 202us) client_negotiation.cc:240] Negotiation successful
0929 03:08:11.772018 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":126,"threads_started":1}
I20260929 03:08:11.774426 22791 heartbeater.cc:347] Connected to a master server at 127.7.209.126:42781
I20260929 03:08:11.774706 22791 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:11.775283 22791 heartbeater.cc:511] Master 127.7.209.126:42781 requested a full tablet report, sending...
I20260929 03:08:11.776398 22333 ts_manager.cc:194] Registered new tserver with Master: 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851)
I20260929 03:08:11.776981 22333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:54299
I20260929 03:08:11.786856 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:08:11.791296 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.787489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49652 (local address 127.7.209.126:42781)
0929 03:08:11.787636 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:11.787641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:11.787846 (+ 205us) server_negotiation.cc:407] Connection header received
0929 03:08:11.788005 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:11.788008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:11.788118 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:11.788202 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:08:11.788405 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.788942 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.789860 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.790095 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.790170 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.790476 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:08:11.790965 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:08:11.791069 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:11.791093 (+ 24us) server_negotiation.cc:299] Negotiation successful
0929 03:08:11.791152 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:08:11.792016 22333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:08:11.796227 22333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:08:11.805680 22455 tablet_service.cc:1467] Processing CreateTablet for tablet ea3f23b7d8d748b7a2e4c553a6e80309 (DEFAULT_TABLE table=test-table [id=0e1368fa75ac4ffaaee702213b77745b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:11.806546 22455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea3f23b7d8d748b7a2e4c553a6e80309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:11.809907 22413 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: Bootstrap starting.
I20260929 03:08:11.810820 22413 tablet_bootstrap.cc:654] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:11.812358 22413 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: No bootstrap required, opened a new log
I20260929 03:08:11.812457 22413 ts_tablet_manager.cc:1397] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260929 03:08:11.812963 22413 raft_consensus.cc:348] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:11.813100 22413 raft_consensus.cc:374] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:11.813148 22413 raft_consensus.cc:729] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b892a39fdc7b44b7a35a9a6418e62fcf, State: Initialized, Role: FOLLOWER
I20260929 03:08:11.813287 22413 consensus_queue.cc:260] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:11.813438 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.805279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:32851 (local address 127.0.0.1:36354)
0929 03:08:11.805842 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:11.805858 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:11.805940 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:11.807015 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:11.807018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:11.807037 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:11.807145 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:11.807150 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.808588 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.808591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:11.809451 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:11.809460 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.812853 (+ 3393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.812857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:11.813024 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.813079 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 03:08:11.813181 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 03:08:11.813289 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":23}
I20260929 03:08:11.813493 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.805945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36354 (local address 127.7.209.67:32851)
0929 03:08:11.806360 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:11.806374 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:11.806403 (+ 29us) server_negotiation.cc:407] Connection header received
0929 03:08:11.806497 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:11.806503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:11.806788 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:11.807044 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:11.807250 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.808447 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.809583 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.812714 (+ 3131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.812842 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.813022 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:11.813215 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 03:08:11.813334 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":187,"threads_started":1}
I20260929 03:08:11.814484 22800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.805500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:44955 (local address 127.0.0.1:59674)
0929 03:08:11.809843 (+ 4343us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:11.809854 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:11.809952 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:11.810502 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:11.810505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:11.810523 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:11.810627 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:11.810632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.811828 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.811831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:11.813019 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:11.813031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.814133 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.814137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:11.814230 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.814255 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 03:08:11.814304 (+ 49us) client_negotiation.cc:240] Negotiation successful
0929 03:08:11.814382 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4281,"mutex_wait_us":27,"thread_start_us":139,"threads_started":1}
I20260929 03:08:11.815106 22413 ts_tablet_manager.cc:1428] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260929 03:08:11.815168 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:11.805897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59674 (local address 127.7.209.66:44955)
0929 03:08:11.806467 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:11.806479 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:11.809968 (+ 3489us) server_negotiation.cc:407] Connection header received
0929 03:08:11.810158 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:11.810164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:11.810367 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:11.810525 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:11.810726 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.811654 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.813120 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:11.813991 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:11.814737 (+ 746us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:11.814872 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:11.814921 (+ 49us) server_negotiation.cc:299] Negotiation successful
0929 03:08:11.815018 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":125,"threads_started":1}
I20260929 03:08:11.817427 22725 tablet_service.cc:1467] Processing CreateTablet for tablet ea3f23b7d8d748b7a2e4c553a6e80309 (DEFAULT_TABLE table=test-table [id=0e1368fa75ac4ffaaee702213b77745b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260929 03:08:11.818130 22522 tablet.cc:2369] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:08:11.817708 22593 tablet_service.cc:1467] Processing CreateTablet for tablet ea3f23b7d8d748b7a2e4c553a6e80309 (DEFAULT_TABLE table=test-table [id=0e1368fa75ac4ffaaee702213b77745b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:11.818756 22725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea3f23b7d8d748b7a2e4c553a6e80309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:11.818778 22593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea3f23b7d8d748b7a2e4c553a6e80309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:11.826051 22804 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Bootstrap starting.
I20260929 03:08:11.827149 22805 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: Bootstrap starting.
I20260929 03:08:11.829457 22804 tablet_bootstrap.cc:654] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:11.830127 22805 tablet_bootstrap.cc:654] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:11.830598 22804 log.cc:824] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:11.831050 22805 log.cc:824] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:11.832721 22805 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: No bootstrap required, opened a new log
I20260929 03:08:11.832762 22804 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: No bootstrap required, opened a new log
I20260929 03:08:11.832939 22805 ts_tablet_manager.cc:1397] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260929 03:08:11.832938 22804 ts_tablet_manager.cc:1397] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 03:08:11.836272 22804 raft_consensus.cc:348] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:11.836270 22805 raft_consensus.cc:348] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:11.836534 22804 raft_consensus.cc:374] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:11.836534 22805 raft_consensus.cc:374] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:11.836570 22804 raft_consensus.cc:729] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2293dbf786864e63b1a063998c33637b, State: Initialized, Role: FOLLOWER
I20260929 03:08:11.836591 22805 raft_consensus.cc:729] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c54f00b42b48ffacfd310fb82c37fe, State: Initialized, Role: FOLLOWER
I20260929 03:08:11.837045 22805 consensus_queue.cc:260] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:11.837045 22804 consensus_queue.cc:260] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:11.837800 22659 heartbeater.cc:503] Master 127.7.209.126:42781 was elected leader, sending a full tablet report...
I20260929 03:08:11.838053 22791 heartbeater.cc:503] Master 127.7.209.126:42781 was elected leader, sending a full tablet report...
I20260929 03:08:11.838125 22805 ts_tablet_manager.cc:1428] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260929 03:08:11.838126 22804 ts_tablet_manager.cc:1428] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 03:08:12.004599 22808 raft_consensus.cc:482] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:08:12.004887 22808 raft_consensus.cc:504] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:12.006379 22808 leader_election.cc:288] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851), b892a39fdc7b44b7a35a9a6418e62fcf (127.7.209.65:32799)
W20260929 03:08:12.007020 22792 tablet.cc:2369] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:08:12.010586 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.006630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:32851 (local address 127.7.209.66:45869)
0929 03:08:12.006774 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:12.006792 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:12.006902 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:12.007275 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:12.007278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:12.007299 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:12.007438 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:12.007445 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:12.008460 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:12.008462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:12.009532 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:12.009541 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:12.010249 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:12.010252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:12.010330 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:12.010351 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 03:08:12.010408 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 03:08:12.010466 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260929 03:08:12.010640 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.006786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:45869 (local address 127.7.209.67:32851)
0929 03:08:12.006928 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:12.006933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:12.006950 (+ 17us) server_negotiation.cc:407] Connection header received
0929 03:08:12.007013 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:12.007016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:12.007074 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:12.007145 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:12.007576 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:12.008369 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:12.009672 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:12.010139 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:12.010258 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:12.010317 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:12.010436 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 03:08:12.010507 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 03:08:12.011293 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.006629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:32799 (local address 127.7.209.66:60181)
0929 03:08:12.007539 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:12.007553 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:12.007686 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:12.008004 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:12.008006 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:12.008020 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:12.008125 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:12.008129 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:12.008946 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:12.008949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:12.009961 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:12.009968 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:12.010895 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:12.010899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:12.010994 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:12.011017 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 03:08:12.011095 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 03:08:12.011163 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"mutex_wait_us":618,"thread_start_us":111,"threads_started":1}
I20260929 03:08:12.011317 22810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.006835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:60181 (local address 127.7.209.65:32799)
0929 03:08:12.007193 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:12.007198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:12.007666 (+ 468us) server_negotiation.cc:407] Connection header received
0929 03:08:12.007785 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:12.007788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:12.007868 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:12.007991 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:12.008220 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:12.008839 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:12.010061 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:12.010698 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:12.010811 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:12.010974 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:12.011106 (+ 132us) server_negotiation.cc:299] Negotiation successful
0929 03:08:12.011191 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":151,"threads_started":1}
I20260929 03:08:12.011368 22745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea3f23b7d8d748b7a2e4c553a6e80309" candidate_uuid: "97c54f00b42b48ffacfd310fb82c37fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2293dbf786864e63b1a063998c33637b" is_pre_election: true
I20260929 03:08:12.011744 22745 raft_consensus.cc:2454] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c54f00b42b48ffacfd310fb82c37fe in term 0.
I20260929 03:08:12.011922 22475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea3f23b7d8d748b7a2e4c553a6e80309" candidate_uuid: "97c54f00b42b48ffacfd310fb82c37fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" is_pre_election: true
I20260929 03:08:12.012248 22475 raft_consensus.cc:2454] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97c54f00b42b48ffacfd310fb82c37fe in term 0.
I20260929 03:08:12.012274 22549 leader_election.cc:302] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2293dbf786864e63b1a063998c33637b, 97c54f00b42b48ffacfd310fb82c37fe; no voters:
I20260929 03:08:12.012575 22808 raft_consensus.cc:2781] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:08:12.012667 22808 raft_consensus.cc:482] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:08:12.012718 22808 raft_consensus.cc:3037] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:12.013949 22808 raft_consensus.cc:504] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:12.014283 22808 leader_election.cc:288] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [CANDIDATE]: Term 1 election: Requested vote from peers 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851), b892a39fdc7b44b7a35a9a6418e62fcf (127.7.209.65:32799)
I20260929 03:08:12.014598 22745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea3f23b7d8d748b7a2e4c553a6e80309" candidate_uuid: "97c54f00b42b48ffacfd310fb82c37fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2293dbf786864e63b1a063998c33637b"
I20260929 03:08:12.014596 22475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea3f23b7d8d748b7a2e4c553a6e80309" candidate_uuid: "97c54f00b42b48ffacfd310fb82c37fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf"
I20260929 03:08:12.014711 22745 raft_consensus.cc:3037] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:12.014765 22475 raft_consensus.cc:3037] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:12.015841 22745 raft_consensus.cc:2454] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c54f00b42b48ffacfd310fb82c37fe in term 1.
I20260929 03:08:12.015842 22475 raft_consensus.cc:2454] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97c54f00b42b48ffacfd310fb82c37fe in term 1.
I20260929 03:08:12.016081 22549 leader_election.cc:302] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2293dbf786864e63b1a063998c33637b, 97c54f00b42b48ffacfd310fb82c37fe; no voters:
I20260929 03:08:12.016247 22808 raft_consensus.cc:2781] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:08:12.016445 22808 raft_consensus.cc:686] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 LEADER]: Becoming Leader. State: Replica: 97c54f00b42b48ffacfd310fb82c37fe, State: Running, Role: LEADER
I20260929 03:08:12.016836 22808 consensus_queue.cc:237] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:12.020673 22332 catalog_manager.cc:5495] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 97c54f00b42b48ffacfd310fb82c37fe (127.7.209.66). New cstate: current_term: 1 leader_uuid: "97c54f00b42b48ffacfd310fb82c37fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } health_report { overall_health: UNKNOWN } } }
W20260929 03:08:12.045559 22660 tablet.cc:2369] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:08:12.055537 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.050706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59684 (local address 127.7.209.66:44955)
0929 03:08:12.050852 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:12.050856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:12.050871 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:08:12.051057 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:12.051060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:12.051143 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:12.051279 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:08:12.051549 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:12.052360 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:12.053647 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:12.054001 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:12.054108 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:12.054464 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:08:12.055114 (+ 650us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:08:12.055236 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:12.055297 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 03:08:12.055371 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 03:08:12.063885 22475 raft_consensus.cc:1261] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Refusing update from remote peer 97c54f00b42b48ffacfd310fb82c37fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:08:12.063910 22745 raft_consensus.cc:1261] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [term 1 FOLLOWER]: Refusing update from remote peer 97c54f00b42b48ffacfd310fb82c37fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:08:12.064661 22808 consensus_queue.cc:1035] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:08:12.064883 22812 consensus_queue.cc:1035] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:08:12.072871 22815 mvcc.cc:205] Tried to move back new op lower bound from 7334507692283592704 to 7334507692103659520. Current Snapshot: MvccSnapshot[applied={T|T < 7334507692283592704}]
I20260929 03:08:12.078852 22816 mvcc.cc:205] Tried to move back new op lower bound from 7334507692283592704 to 7334507692103659520. Current Snapshot: MvccSnapshot[applied={T|T < 7334507692283592704}]
I20260929 03:08:12.081875 22332 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:08:12.082049 22332 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:08:12.084596 22332 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Sending DeleteTablet for 3 replicas of tablet ea3f23b7d8d748b7a2e4c553a6e80309
I20260929 03:08:12.085307 22455 tablet_service.cc:1514] Processing DeleteTablet for tablet ea3f23b7d8d748b7a2e4c553a6e80309 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:12 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:56336
I20260929 03:08:12.085433 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 22371
I20260929 03:08:12.085564 22725 tablet_service.cc:1514] Processing DeleteTablet for tablet ea3f23b7d8d748b7a2e4c553a6e80309 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:12 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:36354
I20260929 03:08:12.086089 22593 tablet_service.cc:1514] Processing DeleteTablet for tablet ea3f23b7d8d748b7a2e4c553a6e80309 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:12 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:59674
I20260929 03:08:12.086575 22824 tablet_replica.cc:330] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: stopping tablet replica
I20260929 03:08:12.086690 22823 tablet_replica.cc:330] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: stopping tablet replica
I20260929 03:08:12.087013 22823 raft_consensus.cc:2229] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:08:12.087014 22824 raft_consensus.cc:2229] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:08:12.087348 22823 raft_consensus.cc:2258] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:12.087556 22824 raft_consensus.cc:2258] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:12.089126 22823 ts_tablet_manager.cc:1905] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:08:12.092279 22823 ts_tablet_manager.cc:1918] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:08:12.092393 22823 log.cc:1196] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/ea3f23b7d8d748b7a2e4c553a6e80309
I20260929 03:08:12.092726 22823 ts_tablet_manager.cc:1939] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Deleting consensus metadata
I20260929 03:08:12.094069 22320 catalog_manager.cc:4841] TS 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851): tablet ea3f23b7d8d748b7a2e4c553a6e80309 (table test-table [id=0e1368fa75ac4ffaaee702213b77745b]) successfully deleted
W20260929 03:08:12.095664 22318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:32799 (error 108)
W20260929 03:08:12.097070 22800 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:08:12.096771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:32799 (local address 127.0.0.1:56348)
0929 03:08:12.096910 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:12.096979 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:32799: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":49}
W20260929 03:08:12.097198 22318 catalog_manager.cc:4584] TS b892a39fdc7b44b7a35a9a6418e62fcf (127.7.209.65:32799): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea3f23b7d8d748b7a2e4c553a6e80309: Network error: Client connection negotiation failed: client connection to 127.7.209.65:32799: connect: Connection reset by peer (error 104)
I20260929 03:08:12.098060 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 22526
I20260929 03:08:12.104923 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 22662
I20260929 03:08:12.105131 22800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.104838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:44955 (local address 127.0.0.1:59694)
0929 03:08:12.104984 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:12.105045 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.66:44955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 03:08:12.111399 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 22300
I20260929 03:08:12.118101 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:42781
--webserver_interface=127.7.209.126
--webserver_port=46451
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:42781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:12.243058 22826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:12.243393 22826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:12.243455 22826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:12.247499 22826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:08:12.247561 22826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:12.247587 22826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:12.247609 22826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:08:12.247629 22826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:08:12.252940 22826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:42781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:42781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=46451
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:12.254184 22826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:12.255424 22826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:12.264719 22831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:12.264765 22826 server_base.cc:1034] running on GCE node
W20260929 03:08:12.264889 22832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:12.264755 22834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:12.265569 22826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:12.266933 22826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:12.268091 22826 hybrid_clock.cc:648] HybridClock initialized: now 1790651292268074 us; error 33 us; skew 500 ppm
Sep 29 03:08:12 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651292, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:12.271294 22826 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:08:12.272464 22826 webserver.cc:460] Webserver started at http://127.7.209.126:46451/ using document root <none> and password file <none>
I20260929 03:08:12.273195 22826 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:12.273299 22826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:12.278285 22826 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 03:08:12.281023 22841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:12.282609 22826 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:12.282734 22826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "e5bc667ab99d40fe9f4a6f856d2b4937"
format_stamp: "Formatted at 2026-09-29 03:08:09 on dist-test-slave-bdps"
server_key: "25026daffd09a902877af35d5b2b79a2"
server_key_iv: "d8673b20d45817dc6194c79945384c11"
server_key_version: "encryptionkey@0"
I20260929 03:08:12.283443 22826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:12.310596 22826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:12.315035 22826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:12.315275 22826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:12.323426 22826 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:42781
I20260929 03:08:12.323701 22893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:42781 every 8 connection(s)
I20260929 03:08:12.324666 22826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:08:12.325042 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 22826
I20260929 03:08:12.325574 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:32799
--local_ip_for_outbound_sockets=127.7.209.65
--tserver_master_addrs=127.7.209.126:42781
--webserver_port=45133
--webserver_interface=127.7.209.65
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:08:12.330760 22894 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:08:12.333179 22894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Bootstrap starting.
I20260929 03:08:12.358584 22894 log.cc:824] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:12.373723 22894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:08:12.374219 22894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937: Bootstrap complete.
I20260929 03:08:12.380286 22894 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } }
I20260929 03:08:12.380842 22894 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5bc667ab99d40fe9f4a6f856d2b4937, State: Initialized, Role: FOLLOWER
I20260929 03:08:12.381487 22894 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } }
I20260929 03:08:12.381677 22894 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:08:12.381763 22894 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:08:12.381863 22894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 2 FOLLOWER]: Advancing to term 3
I20260929 03:08:12.384485 22894 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } }
I20260929 03:08:12.384891 22894 leader_election.cc:302] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5bc667ab99d40fe9f4a6f856d2b4937; no voters:
I20260929 03:08:12.385330 22894 leader_election.cc:288] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 03:08:12.385423 22898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 03:08:12.385722 22898 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [term 3 LEADER]: Becoming Leader. State: Replica: e5bc667ab99d40fe9f4a6f856d2b4937, State: Running, Role: LEADER
I20260929 03:08:12.386034 22898 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } }
I20260929 03:08:12.386790 22894 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:08:12.387023 22898 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5bc667ab99d40fe9f4a6f856d2b4937. Latest consensus state: current_term: 3 leader_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } } }
I20260929 03:08:12.387140 22898 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:12.387428 22899 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5bc667ab99d40fe9f4a6f856d2b4937" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 42781 } } }
I20260929 03:08:12.387492 22899 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5bc667ab99d40fe9f4a6f856d2b4937 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:12.389035 22902 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:08:12.394186 22902 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:08:12.394754 22902 catalog_manager.cc:1259] Loaded cluster ID: 624a79f3d72e482f94d374dd2f2e9bad
I20260929 03:08:12.394807 22902 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 03:08:12.395696 22902 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f603131a9ad google::LogMessage::Fail() at ??:0
@ 0x7f603131d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f603131a4de google::LogMessage::Flush() at ??:0
@ 0x7f603131ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f603bc59fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f603bc5ad2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f603bc57579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f603bc9b640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f603356c3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f603356cc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f603356e3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f603355d512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6034ba16db start_thread at ??:0
@ 0x7f602fe1861f clone at ??:0
W20260929 03:08:12.495985 22896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:12.496272 22896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:12.496315 22896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:12.500330 22896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:12.500385 22896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:12.500443 22896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:08:12.505726 22896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:32799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=45133
--tserver_master_addrs=127.7.209.126:42781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:12.506906 22896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:12.508139 22896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:12.518949 22935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:12.518980 22936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:12.519423 22938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:12.519732 22896 server_base.cc:1034] running on GCE node
I20260929 03:08:12.520195 22896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:12.520890 22896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:12.522043 22896 hybrid_clock.cc:648] HybridClock initialized: now 1790651292522001 us; error 65 us; skew 500 ppm
Sep 29 03:08:12 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651292, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:12.524977 22896 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:08:12.525959 22896 webserver.cc:460] Webserver started at http://127.7.209.65:45133/ using document root <none> and password file <none>
I20260929 03:08:12.526579 22896 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:12.526641 22896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:12.531064 22896 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260929 03:08:12.533517 22945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:12.534970 22896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:08:12.535087 22896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "b892a39fdc7b44b7a35a9a6418e62fcf"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "001814dcaa464b0591a912f9a966cba6"
server_key_iv: "5ff64360c78525aaea871976ccbf5fe0"
server_key_version: "encryptionkey@0"
I20260929 03:08:12.535686 22896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:12.548323 22896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:12.551855 22896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:12.552050 22896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:12.552745 22896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:12.554085 22952 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 03:08:12.560719 22896 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 03:08:12.560787 22896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260929 03:08:12.560861 22896 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 03:08:12.562888 22896 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 03:08:12.562935 22896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.001s
I20260929 03:08:12.563333 22952 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: Bootstrap starting.
I20260929 03:08:12.574453 22896 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:32799
I20260929 03:08:12.574834 23059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:32799 every 8 connection(s)
I20260929 03:08:12.575646 22896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:08:12.585114 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 22896
I20260929 03:08:12.585692 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:44955
--local_ip_for_outbound_sockets=127.7.209.66
--tserver_master_addrs=127.7.209.126:42781
--webserver_port=46577
--webserver_interface=127.7.209.66
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:08:12.589576 22952 log.cc:824] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:12.604354 22952 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:08:12.604839 22952 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: Bootstrap complete.
I20260929 03:08:12.606009 22952 ts_tablet_manager.cc:1397] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: Time spent bootstrapping tablet: real 0.043s user 0.019s sys 0.020s
I20260929 03:08:12.609750 22952 raft_consensus.cc:348] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:12.610320 22952 raft_consensus.cc:729] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b892a39fdc7b44b7a35a9a6418e62fcf, State: Initialized, Role: FOLLOWER
I20260929 03:08:12.610934 22952 consensus_queue.cc:260] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:12.612156 22952 ts_tablet_manager.cc:1428] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
W20260929 03:08:12.679448 23062 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:08:12.578087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:37609)
0929 03:08:12.578728 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:12.578793 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:12.579810 (+ 1017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:12.678414 (+ 98604us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":164,"threads_started":1}
W20260929 03:08:12.680223 23060 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:42781 (0 consecutive failures): Network error: Failed to ping master at 127.7.209.126:42781: Client connection negotiation failed: client connection to 127.7.209.126:42781: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 03:08:12.681231 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.680896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:50321)
0929 03:08:12.681060 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:12.681146 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 03:08:12.681989 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.681759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:46399)
0929 03:08:12.681868 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:12.681922 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 03:08:12.682180 23060 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 03:08:12.719340 23063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:12.719693 23063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:12.719763 23063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:12.724048 23063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:12.724112 23063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:12.724184 23063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:08:12.731428 23063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:44955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=46577
--tserver_master_addrs=127.7.209.126:42781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:12.733096 23063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:12.734783 23063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:12.747421 23073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:12.747437 23071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:12.747552 23063 server_base.cc:1034] running on GCE node
W20260929 03:08:12.747764 23070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:12.748193 23063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:12.748944 23063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:12.750090 23063 hybrid_clock.cc:648] HybridClock initialized: now 1790651292750060 us; error 44 us; skew 500 ppm
Sep 29 03:08:12 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651292, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:12.753151 23063 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:08:12.754186 23063 webserver.cc:460] Webserver started at http://127.7.209.66:46577/ using document root <none> and password file <none>
I20260929 03:08:12.754837 23063 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:12.754901 23063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:12.759409 23063 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:08:12.761780 23080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:12.763167 23063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:12.763278 23063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "97c54f00b42b48ffacfd310fb82c37fe"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "ac5d8dbf82c8bfd59263842af45d265c"
server_key_iv: "25753654f215534ee97560f8d29eec1c"
server_key_version: "encryptionkey@0"
I20260929 03:08:12.763895 23063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:12.794469 23063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:12.798082 23063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:12.798265 23063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:12.798962 23063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:12.800165 23087 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 03:08:12.805111 23063 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 03:08:12.805168 23063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260929 03:08:12.805223 23063 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 03:08:12.807487 23063 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 03:08:12.807548 23063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260929 03:08:12.807976 23087 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: Bootstrap starting.
I20260929 03:08:12.819218 23063 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:44955
I20260929 03:08:12.819339 23194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:44955 every 8 connection(s)
I20260929 03:08:12.820278 23063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:08:12.824220 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.822701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:49177)
0929 03:08:12.823251 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:12.823387 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":134,"threads_started":1}
W20260929 03:08:12.824862 23195 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:42781 (0 consecutive failures): Network error: Failed to ping master at 127.7.209.126:42781: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
I20260929 03:08:12.825435 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.825225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:41117)
0929 03:08:12.825337 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:12.825382 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260929 03:08:12.826074 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.825905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:48879)
0929 03:08:12.825988 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:12.826028 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 03:08:12.826217 23195 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 03:08:12.826324 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 23063
I20260929 03:08:12.826858 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:32851
--local_ip_for_outbound_sockets=127.7.209.67
--tserver_master_addrs=127.7.209.126:42781
--webserver_port=37567
--webserver_interface=127.7.209.67
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:35581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:08:12.828388 23087 log.cc:824] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:12.838629 23087 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:08:12.839093 23087 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: Bootstrap complete.
I20260929 03:08:12.840721 23087 ts_tablet_manager.cc:1397] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.009s
I20260929 03:08:12.844660 23087 raft_consensus.cc:348] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:12.845265 23087 raft_consensus.cc:729] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97c54f00b42b48ffacfd310fb82c37fe, State: Initialized, Role: FOLLOWER
I20260929 03:08:12.845851 23087 consensus_queue.cc:260] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:12.846978 23087 ts_tablet_manager.cc:1428] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe: Time spent starting tablet: real 0.006s user 0.009s sys 0.000s
I20260929 03:08:12.847191 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.846873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:35221)
0929 03:08:12.847015 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:12.847084 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 03:08:12.847870 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:12.847675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:58865)
0929 03:08:12.847775 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:12.847819 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 03:08:12.958401 23199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:12.958688 23199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:12.958731 23199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:12.962764 23199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:12.962832 23199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:12.962889 23199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:08:12.968389 23199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:35581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:32851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=37567
--tserver_master_addrs=127.7.209.126:42781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:12.969717 23199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:12.970965 23199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:12.981957 23206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:12.981966 23208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:12.981962 23205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:12.982321 23199 server_base.cc:1034] running on GCE node
I20260929 03:08:12.982766 23199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:12.983453 23199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:12.984606 23199 hybrid_clock.cc:648] HybridClock initialized: now 1790651292984566 us; error 54 us; skew 500 ppm
Sep 29 03:08:12 dist-test-slave-bdps krb5kdc[21754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651292, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:12.987470 23199 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:08:12.988442 23199 webserver.cc:460] Webserver started at http://127.7.209.67:37567/ using document root <none> and password file <none>
I20260929 03:08:12.989069 23199 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:12.989132 23199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:12.993805 23199 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:12.996260 23215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:12.997727 23199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:12.997844 23199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "2293dbf786864e63b1a063998c33637b"
format_stamp: "Formatted at 2026-09-29 03:08:10 on dist-test-slave-bdps"
server_key: "419789a7f80abc143395002c5f890f18"
server_key_iv: "8e75e06c24098654408c92d7d7ca5f56"
server_key_version: "encryptionkey@0"
I20260929 03:08:12.998492 23199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:13.016975 23199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:13.020110 23199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:13.020344 23199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:13.021098 23199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:13.022099 23199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:13.022146 23199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:13.022192 23199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:13.022219 23199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:13.031980 23199 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:32851
I20260929 03:08:13.032013 23328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:32851 every 8 connection(s)
I20260929 03:08:13.033212 23199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:08:13.035931 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 23199
I20260929 03:08:13.037763 23331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:13.035703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:52643)
0929 03:08:13.036265 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:13.036824 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":123,"threads_started":1}
W20260929 03:08:13.038225 23329 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:42781 (0 consecutive failures): Network error: Failed to ping master at 127.7.209.126:42781: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
I20260929 03:08:13.038780 23331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:13.038586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:52613)
0929 03:08:13.038681 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:13.038727 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260929 03:08:13.039319 23331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:13.039144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:33675)
0929 03:08:13.039228 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:13.039274 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 03:08:13.039458 23329 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 03:08:13.683459 23333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:13.682909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:53419)
0929 03:08:13.683230 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:13.683306 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":116,"threads_started":1}
I20260929 03:08:13.849128 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:13.848651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:44669)
0929 03:08:13.848938 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:13.848999 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":96,"threads_started":1}
I20260929 03:08:13.989028 23335 raft_consensus.cc:482] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:08:13.989293 23335 raft_consensus.cc:504] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:13.990875 23335 leader_election.cc:288] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851), 97c54f00b42b48ffacfd310fb82c37fe (127.7.209.66:44955)
Sep 29 03:08:13 dist-test-slave-bdps krb5kdc[21754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651292, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.66@KRBTEST.COM
Sep 29 03:08:13 dist-test-slave-bdps krb5kdc[21754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651292, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.67@KRBTEST.COM
I20260929 03:08:14.004626 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:13.991433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:44955 (local address 127.7.209.65:52911)
0929 03:08:13.991708 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:13.991721 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:13.991854 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:13.993130 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:13.993137 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:13.993784 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:13.994386 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:13.994403 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:13.995703 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:13.995707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:13.996703 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:13.996712 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:13.997412 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:13.997415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:13.997658 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:13.998297 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:13.998313 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:13.999888 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:14.002038 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:14.002044 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:14.002058 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:14.002386 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:14.002703 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:14.002705 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:14.002721 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:14.002877 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:14.003627 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:14.003632 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:14.003908 (+ 276us) client_negotiation.cc:769] Sending connection context
0929 03:08:14.004086 (+ 178us) client_negotiation.cc:240] Negotiation successful
0929 03:08:14.004403 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":198,"spinlock_wait_cycles":22144,"thread_start_us":85,"threads_started":1}
I20260929 03:08:14.004627 23333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:13.991125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:32851 (local address 127.7.209.65:40923)
0929 03:08:13.991266 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:13.991280 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:13.991396 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:13.993277 (+ 1881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:13.993279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:13.993783 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:13.994386 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:13.994403 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:13.995707 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:13.995710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:13.996700 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:13.996710 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:13.997413 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:13.997416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:13.997658 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:13.998349 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:13.998361 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:14.000192 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:14.002729 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:14.002732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:14.002734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:14.002968 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:14.003360 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:14.003363 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:14.003364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:14.003454 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:14.004093 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:14.004096 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:14.004223 (+ 127us) client_negotiation.cc:769] Sending connection context
0929 03:08:14.004323 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 03:08:14.004479 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":198,"spinlock_wait_cycles":41856}
I20260929 03:08:14.005213 23336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:13.991329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:40923 (local address 127.7.209.67:32851)
0929 03:08:13.991814 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:13.991829 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:13.991876 (+ 47us) server_negotiation.cc:407] Connection header received
0929 03:08:13.992684 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:13.992689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:13.993013 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:13.993323 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:13.994554 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:13.995584 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:13.996811 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:13.997299 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:13.997452 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:14.000383 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:14.000391 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:14.000405 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:14.000437 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:14.002392 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:14.003081 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:14.003086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:14.003092 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:14.003225 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:14.003549 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:14.003553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:14.003555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:14.003992 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:14.004110 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:14.004449 (+ 339us) server_negotiation.cc:299] Negotiation successful
0929 03:08:14.004708 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":165,"threads_started":1}
I20260929 03:08:14.005579 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:13.991646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:52911 (local address 127.7.209.66:44955)
0929 03:08:13.992080 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:13.992091 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:13.992124 (+ 33us) server_negotiation.cc:407] Connection header received
0929 03:08:13.992324 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:13.992330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:13.992747 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:13.993027 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:13.994557 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:13.995571 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:13.996828 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:13.997299 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:13.997454 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:14.000051 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:14.000056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:14.000069 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:14.000095 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:14.001933 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:14.002492 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:14.002497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:14.002501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:14.002636 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:14.002968 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:14.002972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:14.002974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:14.003525 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:14.003685 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:14.004229 (+ 544us) server_negotiation.cc:299] Negotiation successful
0929 03:08:14.004530 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":96,"threads_started":1}
I20260929 03:08:14.006649 23149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea3f23b7d8d748b7a2e4c553a6e80309" candidate_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "97c54f00b42b48ffacfd310fb82c37fe" is_pre_election: true
I20260929 03:08:14.006989 23149 raft_consensus.cc:2454] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b892a39fdc7b44b7a35a9a6418e62fcf in term 1.
W20260929 03:08:14.007867 22949 leader_election.cc:341] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851): Not found: Tablet not found: ea3f23b7d8d748b7a2e4c553a6e80309
I20260929 03:08:14.008109 22949 leader_election.cc:302] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97c54f00b42b48ffacfd310fb82c37fe, b892a39fdc7b44b7a35a9a6418e62fcf; no voters: 2293dbf786864e63b1a063998c33637b
I20260929 03:08:14.008373 23335 raft_consensus.cc:2781] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 03:08:14.008450 23335 raft_consensus.cc:482] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:08:14.008481 23335 raft_consensus.cc:3037] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 1 FOLLOWER]: Advancing to term 2
I20260929 03:08:14.010877 23335 raft_consensus.cc:504] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:14.011283 23335 leader_election.cc:288] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [CANDIDATE]: Term 2 election: Requested vote from peers 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851), 97c54f00b42b48ffacfd310fb82c37fe (127.7.209.66:44955)
I20260929 03:08:14.011669 23149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea3f23b7d8d748b7a2e4c553a6e80309" candidate_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "97c54f00b42b48ffacfd310fb82c37fe"
I20260929 03:08:14.011825 23149 raft_consensus.cc:3037] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 1 FOLLOWER]: Advancing to term 2
W20260929 03:08:14.011914 22949 leader_election.cc:341] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851): Not found: Tablet not found: ea3f23b7d8d748b7a2e4c553a6e80309
I20260929 03:08:14.014029 23149 raft_consensus.cc:2454] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b892a39fdc7b44b7a35a9a6418e62fcf in term 2.
I20260929 03:08:14.014304 22949 leader_election.cc:302] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 97c54f00b42b48ffacfd310fb82c37fe, b892a39fdc7b44b7a35a9a6418e62fcf; no voters: 2293dbf786864e63b1a063998c33637b
I20260929 03:08:14.014458 23335 raft_consensus.cc:2781] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 2 FOLLOWER]: Leader election won for term 2
I20260929 03:08:14.014668 23335 raft_consensus.cc:686] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [term 2 LEADER]: Becoming Leader. State: Replica: b892a39fdc7b44b7a35a9a6418e62fcf, State: Running, Role: LEADER
I20260929 03:08:14.015017 23335 consensus_queue.cc:237] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:14.015348 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:14.015105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:54305)
0929 03:08:14.015233 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:14.015286 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 03:08:14.040355 23341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:14.039955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:34573)
0929 03:08:14.040192 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:14.040252 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":76,"threads_started":1}
W20260929 03:08:14.538944 22949 consensus_peers.cc:487] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf -> Peer 2293dbf786864e63b1a063998c33637b (127.7.209.67:32851): Couldn't send request to peer 2293dbf786864e63b1a063998c33637b. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ea3f23b7d8d748b7a2e4c553a6e80309. This is attempt 1: this message will repeat every 5th retry.
I20260929 03:08:14.575765 23149 raft_consensus.cc:1261] T ea3f23b7d8d748b7a2e4c553a6e80309 P 97c54f00b42b48ffacfd310fb82c37fe [term 2 FOLLOWER]: Refusing update from remote peer b892a39fdc7b44b7a35a9a6418e62fcf: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260929 03:08:14.576591 23342 consensus_queue.cc:1035] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260929 03:08:14.579010 23344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:14.578633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:39753)
0929 03:08:14.578860 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:14.578919 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":96,"threads_started":1}
I20260929 03:08:14.581723 23349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:14.581337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:58741)
0929 03:08:14.581556 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:14.581638 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":58,"threads_started":1}
I20260929 03:08:14.581938 23344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:14.581713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:43273)
0929 03:08:14.581825 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:14.581877 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260929 03:08:14.927147 23350 ts_tablet_manager.cc:927] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Initiating tablet copy from peer b892a39fdc7b44b7a35a9a6418e62fcf (127.7.209.65:32799)
I20260929 03:08:14.927937 23350 tablet_copy_client.cc:316] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: tablet copy: Beginning tablet copy session from remote peer at address 127.7.209.65:32799
Sep 29 03:08:14 dist-test-slave-bdps krb5kdc[21754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651292, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
I20260929 03:08:14.941880 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:14.928539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:32799 (local address 127.7.209.67:44597)
0929 03:08:14.928852 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:14.928882 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:14.929057 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:14.929554 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:14.929560 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:14.930080 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:14.930218 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:14.930224 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:14.931200 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:14.931202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:14.931927 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:14.931935 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:14.932488 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:14.932491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:14.932567 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:14.936993 (+ 4426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:14.937007 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:14.938545 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:14.940345 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:14.940354 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:14.940360 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:14.940584 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:14.940824 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:14.940827 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:14.940828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:14.940896 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:14.941320 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:14.941326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:14.941435 (+ 109us) client_negotiation.cc:769] Sending connection context
0929 03:08:14.941560 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 03:08:14.941731 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":131,"threads_started":1}
I20260929 03:08:14.941936 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:14.928722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:44597 (local address 127.7.209.65:32799)
0929 03:08:14.929071 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:14.929083 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:14.929111 (+ 28us) server_negotiation.cc:407] Connection header received
0929 03:08:14.929164 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:14.929169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:14.929414 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:14.929555 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:14.930319 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:14.931107 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:14.932012 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:14.932408 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:14.932484 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:14.938668 (+ 6184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:14.938677 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:14.938691 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:14.938716 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:14.940253 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:14.940672 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:14.940678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:14.940679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:14.940764 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:14.940953 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:14.940955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:14.940956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:14.941213 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:14.941336 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:14.941658 (+ 322us) server_negotiation.cc:299] Negotiation successful
0929 03:08:14.941798 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":137,"threads_started":1}
I20260929 03:08:14.943014 23034 tablet_copy_service.cc:140] P b892a39fdc7b44b7a35a9a6418e62fcf: Received BeginTabletCopySession request for tablet ea3f23b7d8d748b7a2e4c553a6e80309 from peer 2293dbf786864e63b1a063998c33637b ({username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:44597)
I20260929 03:08:14.943183 23034 tablet_copy_service.cc:161] P b892a39fdc7b44b7a35a9a6418e62fcf: Beginning new tablet copy session on tablet ea3f23b7d8d748b7a2e4c553a6e80309 from peer 2293dbf786864e63b1a063998c33637b at {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:44597: session id = 2293dbf786864e63b1a063998c33637b-ea3f23b7d8d748b7a2e4c553a6e80309
I20260929 03:08:14.944501 23034 tablet_copy_source_session.cc:205] T ea3f23b7d8d748b7a2e4c553a6e80309 P b892a39fdc7b44b7a35a9a6418e62fcf: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 03:08:14.949208 23350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea3f23b7d8d748b7a2e4c553a6e80309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:14.956705 23350 tablet_copy_client.cc:791] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: tablet copy: Starting download of 0 data blocks...
I20260929 03:08:14.956931 23350 tablet_copy_client.cc:655] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: tablet copy: Starting download of 1 WAL segments...
I20260929 03:08:14.958724 23350 tablet_copy_client.cc:531] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 03:08:14.960533 23350 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Bootstrap starting.
I20260929 03:08:14.966132 23350 log.cc:824] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:14.973322 23350 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:08:14.973590 23350 tablet_bootstrap.cc:492] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Bootstrap complete.
I20260929 03:08:14.973934 23350 ts_tablet_manager.cc:1397] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.005s
I20260929 03:08:14.976269 23350 raft_consensus.cc:348] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:14.976506 23350 raft_consensus.cc:729] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2293dbf786864e63b1a063998c33637b, State: Initialized, Role: FOLLOWER
I20260929 03:08:14.976950 23350 consensus_queue.cc:260] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2293dbf786864e63b1a063998c33637b" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 32851 } } peers { permanent_uuid: "97c54f00b42b48ffacfd310fb82c37fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 44955 } } peers { permanent_uuid: "b892a39fdc7b44b7a35a9a6418e62fcf" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 32799 } }
I20260929 03:08:14.977919 23350 ts_tablet_manager.cc:1428] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260929 03:08:14.978088 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:14.977838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:49723)
0929 03:08:14.977955 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:14.978000 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 03:08:14.978578 23034 tablet_copy_service.cc:342] P b892a39fdc7b44b7a35a9a6418e62fcf: Request end of tablet copy session 2293dbf786864e63b1a063998c33637b-ea3f23b7d8d748b7a2e4c553a6e80309 received from {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:44597
I20260929 03:08:14.978683 23034 tablet_copy_service.cc:433] P b892a39fdc7b44b7a35a9a6418e62fcf: ending tablet copy session 2293dbf786864e63b1a063998c33637b-ea3f23b7d8d748b7a2e4c553a6e80309 on tablet ea3f23b7d8d748b7a2e4c553a6e80309 with peer 2293dbf786864e63b1a063998c33637b
I20260929 03:08:14.978716 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:14.978501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:58649)
0929 03:08:14.978600 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:14.978646 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 03:08:15.360219 23358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:15.359773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:54745)
0929 03:08:15.360045 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:15.360099 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":84,"threads_started":1}
I20260929 03:08:15.360340 23283 raft_consensus.cc:1203] T ea3f23b7d8d748b7a2e4c553a6e80309 P 2293dbf786864e63b1a063998c33637b [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260929 03:08:15.361481 23352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:15.361202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:50741)
0929 03:08:15.361331 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:15.361394 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 03:08:15.582991 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:15.582563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:44737)
0929 03:08:15.582827 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:15.582887 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":100,"threads_started":1}
I20260929 03:08:16.361702 23360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:16.361135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:54025)
0929 03:08:16.361466 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:16.361540 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":114,"threads_started":1}
I20260929 03:08:16.362681 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:16.362222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:42613)
0929 03:08:16.362508 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:16.362565 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":101,"threads_started":1}
I20260929 03:08:16.584439 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:16.583832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:51119)
0929 03:08:16.584221 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:16.584322 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":126,"threads_started":1}
I20260929 03:08:17.363094 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:17.362546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:54885)
0929 03:08:17.362875 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:17.362965 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":120,"threads_started":1}
I20260929 03:08:17.363873 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:17.363408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:52381)
0929 03:08:17.363716 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:17.363775 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":92,"threads_started":1}
I20260929 03:08:17.585772 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:17.585212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:57465)
0929 03:08:17.585531 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:17.585598 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":145,"threads_started":1}
I20260929 03:08:18.364807 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:18.364247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:47025)
0929 03:08:18.364572 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:18.364654 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":123,"threads_started":1}
I20260929 03:08:18.365298 23368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:18.364839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:42967)
0929 03:08:18.365142 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:18.365197 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":103,"threads_started":1}
I20260929 03:08:18.365664 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:18.365405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:41213)
0929 03:08:18.365518 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:18.365567 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260929 03:08:18.587145 23369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:18.586673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:55399)
0929 03:08:18.586943 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:18.587017 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":89,"threads_started":1}
I20260929 03:08:19.366669 23370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:19.366177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:35049)
0929 03:08:19.366480 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:19.366545 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":107,"threads_started":1}
I20260929 03:08:19.366778 23371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:19.366367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:51307)
0929 03:08:19.366625 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:19.366680 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":83,"threads_started":1}
I20260929 03:08:19.588495 23372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:19.587992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:44767)
0929 03:08:19.588306 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:19.588377 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":98,"threads_started":1}
I20260929 03:08:20.368331 23374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:20.367757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:38741)
0929 03:08:20.368157 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:20.368214 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":105,"threads_started":1}
I20260929 03:08:20.368301 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:20.367718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:44935)
0929 03:08:20.368066 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:20.368141 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":137,"threads_started":1}
I20260929 03:08:20.589840 23375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:20.589283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:39049)
0929 03:08:20.589653 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:20.589722 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":128,"threads_started":1}
I20260929 03:08:21.369690 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:21.369178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:38069)
0929 03:08:21.369473 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:21.369535 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":105,"threads_started":1}
I20260929 03:08:21.369760 23377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:21.369178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:47517)
0929 03:08:21.369560 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:21.369650 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":105,"threads_started":1}
I20260929 03:08:21.591089 23378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:21.590599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:35447)
0929 03:08:21.590901 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:21.590967 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":121,"threads_started":1}
I20260929 03:08:22.371131 23379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:22.370558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:38935)
0929 03:08:22.370940 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:22.371004 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":109,"threads_started":1}
I20260929 03:08:22.371188 23380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:22.370574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:34751)
0929 03:08:22.371002 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:22.371073 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":115,"threads_started":1}
I20260929 03:08:22.592432 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:22.591918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:48967)
0929 03:08:22.592221 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:22.592305 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":96,"threads_started":1}
I20260929 03:08:23.372643 23382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:23.372168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:43817)
0929 03:08:23.372461 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:23.372522 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":98,"threads_started":1}
I20260929 03:08:23.372665 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:23.372145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:47737)
0929 03:08:23.372514 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:23.372568 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":109,"threads_started":1}
I20260929 03:08:23.593904 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:23.593393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:57613)
0929 03:08:23.593709 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:23.593779 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":96,"threads_started":1}
I20260929 03:08:24.374044 23385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:24.373521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:57581)
0929 03:08:24.373844 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:24.373914 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":131,"threads_started":1}
I20260929 03:08:24.374089 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:24.373521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:34389)
0929 03:08:24.373930 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:24.373989 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":132,"threads_started":1}
I20260929 03:08:24.595472 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:24.594838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:46113)
0929 03:08:24.595188 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:24.595313 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":121,"threads_started":1}
I20260929 03:08:25.375357 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:25.374872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:46533)
0929 03:08:25.375171 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:25.375234 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":112,"threads_started":1}
I20260929 03:08:25.375357 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:25.374857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:34473)
0929 03:08:25.375172 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:25.375246 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":98,"threads_started":1}
I20260929 03:08:25.596907 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:25.596358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:38849)
0929 03:08:25.596698 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:25.596779 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":98,"threads_started":1}
I20260929 03:08:26.376664 23391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:26.376166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:34797)
0929 03:08:26.376484 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:26.376544 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":113,"threads_started":1}
I20260929 03:08:26.376685 23392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:26.376239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:57745)
0929 03:08:26.376530 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:26.376588 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":100,"threads_started":1}
I20260929 03:08:26.598362 23393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:26.597841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:47105)
0929 03:08:26.598152 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:26.598225 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":106,"threads_started":1}
I20260929 03:08:27.378003 23394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:27.377455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.65:37291)
0929 03:08:27.377784 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:27.377859 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":92,"threads_started":1}
I20260929 03:08:27.378047 23395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:27.377535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.67:51571)
0929 03:08:27.377882 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:27.377946 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":136,"threads_started":1}
I20260929 03:08:27.599681 23396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:27.599188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:42781 (local address 127.7.209.66:49837)
0929 03:08:27.599485 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:27.599560 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:42781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":100,"threads_started":1}
I20260929 03:08:28.040439 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 22896
I20260929 03:08:28.046916 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 23063
I20260929 03:08:28.052457 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 23199
2026-09-29T03:08:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19470 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:08:28 dist-test-slave-bdps krb5kdc[23399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:08:28 dist-test-slave-bdps krb5kdc[23399](info): set up 2 sockets
Sep 29 03:08:28 dist-test-slave-bdps krb5kdc[23399](info): commencing operation
krb5kdc: starting...
W20260929 03:08:29.014108 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:08:29 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:08:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:08:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:08:29.143908 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:44437
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:44437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:29.267208 23412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:29.267485 23412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:29.267524 23412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:29.271616 23412 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 03:08:29.271670 23412 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 03:08:29.271683 23412 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 03:08:29.271695 23412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:08:29.271713 23412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:29.271729 23412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:29.271750 23412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:08:29.271780 23412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:08:29.276942 23412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:44437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:44437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:29.278210 23412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:29.279421 23412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:08:29.289155 23412 server_base.cc:1034] running on GCE node
W20260929 03:08:29.289250 23420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:29.289170 23417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:29.289397 23418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:29.290060 23412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:29.291340 23412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:29.292537 23412 hybrid_clock.cc:648] HybridClock initialized: now 1790651309292520 us; error 32 us; skew 500 ppm
Sep 29 03:08:29 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651309, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:29.295764 23412 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:08:29.296841 23412 webserver.cc:460] Webserver started at http://127.7.209.126:33709/ using document root <none> and password file <none>
I20260929 03:08:29.297498 23412 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:29.297570 23412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:29.297824 23412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:08:29.300237 23412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "a2165666b19148f2b38182ada4a15483"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "a2571d246946e08b5124f66990c7d828"
server_key_iv: "49e6817a6e0d6ade62deab758cd052fb"
server_key_version: "encryptionkey@0"
I20260929 03:08:29.300843 23412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "a2165666b19148f2b38182ada4a15483"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "a2571d246946e08b5124f66990c7d828"
server_key_iv: "49e6817a6e0d6ade62deab758cd052fb"
server_key_version: "encryptionkey@0"
I20260929 03:08:29.305192 23412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 03:08:29.308459 23427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:29.309983 23412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:29.310117 23412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "a2165666b19148f2b38182ada4a15483"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "a2571d246946e08b5124f66990c7d828"
server_key_iv: "49e6817a6e0d6ade62deab758cd052fb"
server_key_version: "encryptionkey@0"
I20260929 03:08:29.310238 23412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:29.333400 23412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:29.336858 23412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:29.337070 23412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:29.344990 23412 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:44437
I20260929 03:08:29.345287 23479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:44437 every 8 connection(s)
I20260929 03:08:29.346125 23412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:08:29.349452 23480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:29.350878 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 23412
I20260929 03:08:29.351001 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:08:29.351402 8005 external_mini_cluster.cc:1428] Setting key 887d370e436ccaa17b0edc43baedf202
I20260929 03:08:29.357082 23480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Bootstrap starting.
I20260929 03:08:29.360618 23480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Neither blocks nor log segments found. Creating new log.
Sep 29 03:08:29 dist-test-slave-bdps krb5kdc[23399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:29.361763 23480 log.cc:824] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:29.364367 23480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: No bootstrap required, opened a new log
I20260929 03:08:29.367313 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:29.352987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39396 (local address 127.7.209.126:44437)
0929 03:08:29.353422 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:29.353443 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:29.353484 (+ 41us) server_negotiation.cc:407] Connection header received
0929 03:08:29.354260 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:29.354283 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:29.354697 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:29.355050 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:29.356085 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:29.357279 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:29.358526 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:29.359074 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:29.359207 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:29.362389 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:29.362395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:29.362414 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:29.362446 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:29.364135 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:29.364627 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:29.364632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:29.364637 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:29.364761 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:29.365031 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:29.365035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:29.365037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:29.365519 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:29.365760 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:29.366073 (+ 313us) server_negotiation.cc:299] Negotiation successful
0929 03:08:29.366425 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":123,"threads_started":1}
I20260929 03:08:29.368711 23480 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } }
I20260929 03:08:29.369040 23480 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:29.369099 23480 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2165666b19148f2b38182ada4a15483, State: Initialized, Role: FOLLOWER
I20260929 03:08:29.369654 23480 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } }
I20260929 03:08:29.369843 23480 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:08:29.369915 23480 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:08:29.370020 23480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:29.371088 23480 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } }
I20260929 03:08:29.371464 23480 leader_election.cc:302] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2165666b19148f2b38182ada4a15483; no voters:
I20260929 03:08:29.371845 23480 leader_election.cc:288] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:08:29.371937 23485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:08:29.372164 23485 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 1 LEADER]: Becoming Leader. State: Replica: a2165666b19148f2b38182ada4a15483, State: Running, Role: LEADER
I20260929 03:08:29.372496 23485 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } }
I20260929 03:08:29.373396 23480 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:08:29.374442 23487 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2165666b19148f2b38182ada4a15483. Latest consensus state: current_term: 1 leader_uuid: "a2165666b19148f2b38182ada4a15483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } } }
I20260929 03:08:29.374576 23487 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:29.375026 23494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:08:29.375003 23486 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2165666b19148f2b38182ada4a15483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } } }
I20260929 03:08:29.375202 23486 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:29.379451 23494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:08:29.385828 23494 catalog_manager.cc:1347] Generated new cluster ID: 2a0a08c511d440c08ecd16f81f520fc4
I20260929 03:08:29.385890 23494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:08:29.406270 23494 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:08:29.407466 23494 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:08:29.418377 23494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Generated new TSK 0
I20260929 03:08:29.419291 23494 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:08:29.460400 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:44437
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:29.584733 23506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:29.585094 23506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:29.585143 23506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:29.589792 23506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:29.589905 23506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:29.589975 23506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:08:29.595871 23506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:44437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:29.597359 23506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:29.598651 23506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:29.609853 23514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:29.609872 23511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:29.609877 23512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:29.610123 23506 server_base.cc:1034] running on GCE node
I20260929 03:08:29.610646 23506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:29.611370 23506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:29.612556 23506 hybrid_clock.cc:648] HybridClock initialized: now 1790651309612537 us; error 30 us; skew 500 ppm
Sep 29 03:08:29 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651309, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:29.615849 23506 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:08:29.616794 23506 webserver.cc:460] Webserver started at http://127.7.209.65:42903/ using document root <none> and password file <none>
I20260929 03:08:29.617435 23506 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:29.617498 23506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:29.617748 23506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:08:29.620030 23506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "1e7d84ebe03848b3a53f547cd9890029"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "f50a216f29e5dca3b7f41f6d8209a0e7"
server_key_iv: "43350602b7230a4614fda72b90557deb"
server_key_version: "encryptionkey@0"
I20260929 03:08:29.620595 23506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "1e7d84ebe03848b3a53f547cd9890029"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "f50a216f29e5dca3b7f41f6d8209a0e7"
server_key_iv: "43350602b7230a4614fda72b90557deb"
server_key_version: "encryptionkey@0"
I20260929 03:08:29.624624 23506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 03:08:29.627226 23521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:29.628576 23506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 03:08:29.628691 23506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "1e7d84ebe03848b3a53f547cd9890029"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "f50a216f29e5dca3b7f41f6d8209a0e7"
server_key_iv: "43350602b7230a4614fda72b90557deb"
server_key_version: "encryptionkey@0"
I20260929 03:08:29.628809 23506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:29.664532 23506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:29.667934 23506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:29.668130 23506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:29.668845 23506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:29.669840 23506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:29.669890 23506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:29.669938 23506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:29.669965 23506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:29.680133 23506 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:33631
I20260929 03:08:29.680159 23634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:33631 every 8 connection(s)
I20260929 03:08:29.681254 23506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:08:29.688113 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 23506
I20260929 03:08:29.688266 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:08:29.688619 8005 external_mini_cluster.cc:1428] Setting key df200b4503cff6899dde3547a8238acd
Sep 29 03:08:29 dist-test-slave-bdps krb5kdc[23399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651309, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:29.696445 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:29.683439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:53701 (local address 127.7.209.126:44437)
0929 03:08:29.683583 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:29.683586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:29.684832 (+ 1246us) server_negotiation.cc:407] Connection header received
0929 03:08:29.685925 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:29.685929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:29.685998 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:29.686083 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:29.687460 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:29.688181 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:29.689663 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:29.689919 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:29.690021 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:29.692908 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:29.692912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:29.692914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:29.692938 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:29.694395 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:29.694897 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:29.694901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:29.694902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:29.694986 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:29.695307 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:29.695310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:29.695313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:29.695503 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:29.695592 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:29.696120 (+ 528us) server_negotiation.cc:299] Negotiation successful
0929 03:08:29.696264 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 03:08:29.697371 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:29.683914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:53701)
0929 03:08:29.684654 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:29.684701 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:29.685682 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:29.686315 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:29.686323 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:29.686730 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:29.687310 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:29.687326 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:29.688335 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:29.688340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:29.689488 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:29.689500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:29.690087 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:29.690092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:29.690326 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:29.690982 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:29.691002 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:29.692731 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:29.694504 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:29.694510 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:29.694527 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:29.694805 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:29.695066 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:29.695069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:29.695072 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:29.695221 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:29.695598 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:29.695606 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:29.695883 (+ 277us) client_negotiation.cc:769] Sending connection context
0929 03:08:29.696064 (+ 181us) client_negotiation.cc:240] Negotiation successful
0929 03:08:29.696420 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":132,"threads_started":1}
I20260929 03:08:29.698670 23635 heartbeater.cc:347] Connected to a master server at 127.7.209.126:44437
I20260929 03:08:29.698913 23635 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:08:29.699450 23635 heartbeater.cc:511] Master 127.7.209.126:44437 requested a full tablet report, sending...
I20260929 03:08:29.701107 23444 ts_manager.cc:194] Registered new tserver with Master: 1e7d84ebe03848b3a53f547cd9890029 (127.7.209.65:33631)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
I20260929 03:08:29.702487 23444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:53701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:08:29.716332 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:44437
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:29.838160 23640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:29.838459 23640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:29.838506 23640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:29.842509 23640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:29.842584 23640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:29.842660 23640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:08:29.847962 23640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:44437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:29.849143 23640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:29.850347 23640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:29.860870 23648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:29.860925 23645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:29.860895 23640 server_base.cc:1034] running on GCE node
W20260929 03:08:29.860945 23646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:29.861486 23640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:29.862188 23640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:29.863361 23640 hybrid_clock.cc:648] HybridClock initialized: now 1790651309863325 us; error 40 us; skew 500 ppm
Sep 29 03:08:29 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651309, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:29.866254 23640 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:08:29.867278 23640 webserver.cc:460] Webserver started at http://127.7.209.66:40915/ using document root <none> and password file <none>
I20260929 03:08:29.867889 23640 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:29.867949 23640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:29.868144 23640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:08:29.870522 23640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "7b59d87795354e81adada2376afdd3fb"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "a20dcb298b767972495ffcd197105449"
server_key_iv: "cc02bc3c439a7679ea1249b53d49b199"
server_key_version: "encryptionkey@0"
I20260929 03:08:29.871183 23640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "7b59d87795354e81adada2376afdd3fb"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "a20dcb298b767972495ffcd197105449"
server_key_iv: "cc02bc3c439a7679ea1249b53d49b199"
server_key_version: "encryptionkey@0"
I20260929 03:08:29.875733 23640 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:08:29.879309 23655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:29.880940 23640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:08:29.881076 23640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "7b59d87795354e81adada2376afdd3fb"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "a20dcb298b767972495ffcd197105449"
server_key_iv: "cc02bc3c439a7679ea1249b53d49b199"
server_key_version: "encryptionkey@0"
I20260929 03:08:29.881222 23640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:29.894134 23640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:29.897689 23640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:29.897891 23640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:29.898561 23640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:29.899510 23640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:29.899560 23640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:29.899616 23640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:29.899644 23640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:29.909075 23640 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:39239
I20260929 03:08:29.909127 23768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:39239 every 8 connection(s)
I20260929 03:08:29.910110 23640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:08:29.913991 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 23640
I20260929 03:08:29.914129 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:08:29.914444 8005 external_mini_cluster.cc:1428] Setting key 8827e103a15c53586375d6fbbd3a7e63
Sep 29 03:08:29 dist-test-slave-bdps krb5kdc[23399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651309, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:29.926079 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:29.912142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:53711 (local address 127.7.209.126:44437)
0929 03:08:29.912294 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:29.912298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:29.913473 (+ 1175us) server_negotiation.cc:407] Connection header received
0929 03:08:29.914630 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:29.914634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:29.914708 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:29.914806 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:29.916423 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:29.917189 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:29.918637 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:29.918946 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:29.919038 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:29.922148 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:29.922152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:29.922155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:29.922192 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:29.923817 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:29.924393 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:29.924397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:29.924399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:29.924503 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:29.924822 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:29.924826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:29.924828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:29.925082 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:29.925184 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:29.925740 (+ 556us) server_negotiation.cc:299] Negotiation successful
0929 03:08:29.925883 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:08:29.927048 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:29.912564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:53711)
0929 03:08:29.913293 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:29.913341 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:29.914379 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:29.915036 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:29.915046 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:29.915536 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:29.916264 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:29.916284 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:29.917323 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:29.917325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:29.918418 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:29.918428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:29.919131 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:29.919135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:29.919370 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:29.920116 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:29.920139 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:29.921961 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:29.923953 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:29.923960 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:29.923982 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:29.924285 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:29.924585 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:29.924587 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:29.924588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:29.924721 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:29.925172 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:29.925177 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:29.925446 (+ 269us) client_negotiation.cc:769] Sending connection context
0929 03:08:29.925652 (+ 206us) client_negotiation.cc:240] Negotiation successful
0929 03:08:29.925986 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":132,"threads_started":1}
I20260929 03:08:29.928412 23769 heartbeater.cc:347] Connected to a master server at 127.7.209.126:44437
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
I20260929 03:08:29.928709 23769 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:29.929318 23769 heartbeater.cc:511] Master 127.7.209.126:44437 requested a full tablet report, sending...
I20260929 03:08:29.930589 23444 ts_manager.cc:194] Registered new tserver with Master: 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239)
I20260929 03:08:29.931236 23444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:53711
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:08:29.942866 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:44437
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:30.066094 23774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:30.066380 23774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:30.066437 23774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:30.070434 23774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:30.070497 23774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:30.070574 23774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:08:30.076077 23774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:44437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:30.077390 23774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:30.078677 23774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:30.089460 23779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:30.089475 23780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:30.089563 23774 server_base.cc:1034] running on GCE node
W20260929 03:08:30.089470 23782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:30.090044 23774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:30.090771 23774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:30.091933 23774 hybrid_clock.cc:648] HybridClock initialized: now 1790651310091916 us; error 32 us; skew 500 ppm
Sep 29 03:08:30 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651310, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:30.094843 23774 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:08:30.095717 23774 webserver.cc:460] Webserver started at http://127.7.209.67:33741/ using document root <none> and password file <none>
I20260929 03:08:30.096328 23774 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:30.096395 23774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:30.096611 23774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:08:30.098955 23774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "50ac64f16ffb4ae88ac445862599dafa"
format_stamp: "Formatted at 2026-09-29 03:08:30 on dist-test-slave-bdps"
server_key: "bedb3696135ca42472ac86cb9f68416e"
server_key_iv: "31dfd8c02cb999a8f273c7c7811069cb"
server_key_version: "encryptionkey@0"
I20260929 03:08:30.099539 23774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "50ac64f16ffb4ae88ac445862599dafa"
format_stamp: "Formatted at 2026-09-29 03:08:30 on dist-test-slave-bdps"
server_key: "bedb3696135ca42472ac86cb9f68416e"
server_key_iv: "31dfd8c02cb999a8f273c7c7811069cb"
server_key_version: "encryptionkey@0"
I20260929 03:08:30.103525 23774 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:08:30.106025 23789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:30.107336 23774 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 03:08:30.107440 23774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "50ac64f16ffb4ae88ac445862599dafa"
format_stamp: "Formatted at 2026-09-29 03:08:30 on dist-test-slave-bdps"
server_key: "bedb3696135ca42472ac86cb9f68416e"
server_key_iv: "31dfd8c02cb999a8f273c7c7811069cb"
server_key_version: "encryptionkey@0"
I20260929 03:08:30.107532 23774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:30.120982 23774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:30.124328 23774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:30.124511 23774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:30.125182 23774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:30.126154 23774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:30.126202 23774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:30.126259 23774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:30.126289 23774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:30.135630 23774 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:33065
I20260929 03:08:30.135651 23902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:33065 every 8 connection(s)
I20260929 03:08:30.136550 23774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:08:30.139032 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 23774
I20260929 03:08:30.139171 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:08:30.139572 8005 external_mini_cluster.cc:1428] Setting key 94f11cbc39768e0e5886ace1b5426b44
Sep 29 03:08:30 dist-test-slave-bdps krb5kdc[23399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651310, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:30.158792 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.138400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:38473 (local address 127.7.209.126:44437)
0929 03:08:30.138567 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:30.138572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:30.139767 (+ 1195us) server_negotiation.cc:407] Connection header received
0929 03:08:30.140829 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:30.140832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:30.140898 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:30.140978 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:30.142302 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.142868 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.144327 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.144680 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.144801 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.153430 (+ 8629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:30.153434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:30.153437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:30.153467 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:30.155422 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:30.156802 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:30.156806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:30.156809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:30.156918 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:30.157321 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:30.157325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:30.157327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:30.157631 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:30.157745 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:30.158421 (+ 676us) server_negotiation.cc:299] Negotiation successful
0929 03:08:30.158581 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 03:08:30.159884 23905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.138828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:38473)
0929 03:08:30.139594 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:30.139655 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:30.140595 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:30.141171 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:30.141179 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:30.141532 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:30.142174 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:30.142189 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.142964 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.142967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:30.144201 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:30.144211 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.144827 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.144830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:30.145020 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.145557 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:30.145572 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:30.153270 (+ 7698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:30.156297 (+ 3027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:30.156304 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:30.156325 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:30.156702 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:30.157015 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:30.157018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:30.157021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:30.157216 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:30.157747 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:30.157755 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:30.158117 (+ 362us) client_negotiation.cc:769] Sending connection context
0929 03:08:30.158370 (+ 253us) client_negotiation.cc:240] Negotiation successful
0929 03:08:30.158799 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":118,"threads_started":1}
I20260929 03:08:30.161165 23903 heartbeater.cc:347] Connected to a master server at 127.7.209.126:44437
I20260929 03:08:30.161459 23903 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:30.162133 23903 heartbeater.cc:511] Master 127.7.209.126:44437 requested a full tablet report, sending...
I20260929 03:08:30.163311 23444 ts_manager.cc:194] Registered new tserver with Master: 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065)
I20260929 03:08:30.163964 23444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:38473
I20260929 03:08:30.166404 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:08:30.175767 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.168730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39408 (local address 127.7.209.126:44437)
0929 03:08:30.168862 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:30.168866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:30.169060 (+ 194us) server_negotiation.cc:407] Connection header received
0929 03:08:30.169169 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:30.169171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:30.169211 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:30.169279 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:30.170044 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.170585 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.171370 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.171575 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.171646 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.172974 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:30.172976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:30.172978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:30.172993 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:30.174335 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:30.174714 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:30.174716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:30.174718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:30.174803 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:30.174988 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:30.174990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:30.174991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:30.175235 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:30.175343 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:30.175511 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 03:08:30.175617 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:08:30.178668 23444 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:08:30.182614 23444 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:08:30.199689 23917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.193768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:33065 (local address 127.0.0.1:44168)
0929 03:08:30.194059 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:30.194072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:30.194163 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:30.195526 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:30.195529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:30.195547 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:30.195641 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:30.195646 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.196962 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.196966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:30.197865 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:30.197875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.199071 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.199075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:30.199182 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.199224 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 03:08:30.199355 (+ 131us) client_negotiation.cc:240] Negotiation successful
0929 03:08:30.199507 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":4,"thread_start_us":116,"threads_started":1}
I20260929 03:08:30.199702 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.194128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44168 (local address 127.7.209.67:33065)
0929 03:08:30.194539 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:30.194550 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:30.194575 (+ 25us) server_negotiation.cc:407] Connection header received
0929 03:08:30.194658 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:30.194663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:30.195355 (+ 692us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:30.195781 (+ 426us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:30.195958 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.196838 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.197997 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.198942 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.199059 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.199240 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:30.199400 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 03:08:30.199515 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":183,"threads_started":1}
I20260929 03:08:30.199891 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.192822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:39239 (local address 127.0.0.1:36960)
0929 03:08:30.193374 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:30.193438 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:30.193591 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:30.194637 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:30.194645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:30.194670 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:30.194867 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:30.194875 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.196406 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.196410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:30.197959 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:30.197965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.199323 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.199327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:30.199498 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.199536 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 03:08:30.199606 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 03:08:30.199698 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":183,"thread_start_us":113,"threads_started":1}
I20260929 03:08:30.199961 23918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.193901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54202 (local address 127.7.209.65:33631)
0929 03:08:30.194382 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:30.194400 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:30.194427 (+ 27us) server_negotiation.cc:407] Connection header received
0929 03:08:30.194517 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:30.194523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:30.194722 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:30.194885 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:30.195141 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.196070 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.198415 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.199032 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.199137 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.199288 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:30.199696 (+ 408us) server_negotiation.cc:299] Negotiation successful
0929 03:08:30.199813 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":145,"threads_started":1}
I20260929 03:08:30.200299 23916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.193083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:33631 (local address 127.0.0.1:54202)
0929 03:08:30.193526 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:30.193540 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:30.193688 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:30.194887 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:30.194891 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:30.194907 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:30.195020 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:30.195027 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.196199 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.196203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:30.197950 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:30.197961 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.199323 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.199327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:30.199530 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.199549 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 03:08:30.199606 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 03:08:30.200162 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":634,"thread_start_us":116,"threads_started":1}
I20260929 03:08:30.200315 23915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.192989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36960 (local address 127.7.209.66:39239)
0929 03:08:30.193993 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:30.194007 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:30.194039 (+ 32us) server_negotiation.cc:407] Connection header received
0929 03:08:30.194143 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:30.194150 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:30.194442 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:30.194637 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:30.195025 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.196261 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.198129 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.198942 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.199059 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.199251 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:30.200044 (+ 793us) server_negotiation.cc:299] Negotiation successful
0929 03:08:30.200164 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":198,"threads_started":1}
I20260929 03:08:30.203619 23837 tablet_service.cc:1467] Processing CreateTablet for tablet 5762effdd06443749a860fcf5622a933 (DEFAULT_TABLE table=test-table [id=bd0f0516eabd4fe486e4e20acaefc0aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:30.203619 23569 tablet_service.cc:1467] Processing CreateTablet for tablet 5762effdd06443749a860fcf5622a933 (DEFAULT_TABLE table=test-table [id=bd0f0516eabd4fe486e4e20acaefc0aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:30.203622 23703 tablet_service.cc:1467] Processing CreateTablet for tablet 5762effdd06443749a860fcf5622a933 (DEFAULT_TABLE table=test-table [id=bd0f0516eabd4fe486e4e20acaefc0aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:30.204859 23703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5762effdd06443749a860fcf5622a933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:30.204870 23837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5762effdd06443749a860fcf5622a933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:30.204887 23569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5762effdd06443749a860fcf5622a933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:30.212113 23920 tablet_bootstrap.cc:492] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa: Bootstrap starting.
I20260929 03:08:30.213047 23922 tablet_bootstrap.cc:492] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Bootstrap starting.
I20260929 03:08:30.213773 23921 tablet_bootstrap.cc:492] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Bootstrap starting.
I20260929 03:08:30.215495 23922 tablet_bootstrap.cc:654] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:30.215533 23920 tablet_bootstrap.cc:654] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:30.215885 23921 tablet_bootstrap.cc:654] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:30.216336 23922 log.cc:824] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:30.216365 23920 log.cc:824] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:30.216627 23921 log.cc:824] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:30.218336 23922 tablet_bootstrap.cc:492] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: No bootstrap required, opened a new log
I20260929 03:08:30.218408 23921 tablet_bootstrap.cc:492] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: No bootstrap required, opened a new log
I20260929 03:08:30.218408 23920 tablet_bootstrap.cc:492] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa: No bootstrap required, opened a new log
I20260929 03:08:30.218539 23922 ts_tablet_manager.cc:1397] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 03:08:30.218573 23921 ts_tablet_manager.cc:1397] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 03:08:30.218573 23920 ts_tablet_manager.cc:1397] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260929 03:08:30.221594 23922 raft_consensus.cc:348] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
I20260929 03:08:30.221873 23922 raft_consensus.cc:374] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:30.221946 23922 raft_consensus.cc:729] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b59d87795354e81adada2376afdd3fb, State: Initialized, Role: FOLLOWER
I20260929 03:08:30.222401 23922 consensus_queue.cc:260] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
I20260929 03:08:30.222532 23920 raft_consensus.cc:348] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
I20260929 03:08:30.222865 23920 raft_consensus.cc:374] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:30.222954 23920 raft_consensus.cc:729] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ac64f16ffb4ae88ac445862599dafa, State: Initialized, Role: FOLLOWER
I20260929 03:08:30.223049 23769 heartbeater.cc:503] Master 127.7.209.126:44437 was elected leader, sending a full tablet report...
I20260929 03:08:30.223067 23921 raft_consensus.cc:348] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
I20260929 03:08:30.223301 23921 raft_consensus.cc:374] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:30.223353 23921 raft_consensus.cc:729] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7d84ebe03848b3a53f547cd9890029, State: Initialized, Role: FOLLOWER
I20260929 03:08:30.223413 23922 ts_tablet_manager.cc:1428] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260929 03:08:30.223472 23920 consensus_queue.cc:260] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
I20260929 03:08:30.223728 23921 consensus_queue.cc:260] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
I20260929 03:08:30.224105 23903 heartbeater.cc:503] Master 127.7.209.126:44437 was elected leader, sending a full tablet report...
I20260929 03:08:30.224597 23635 heartbeater.cc:503] Master 127.7.209.126:44437 was elected leader, sending a full tablet report...
I20260929 03:08:30.225433 23920 ts_tablet_manager.cc:1428] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260929 03:08:30.224617 23921 ts_tablet_manager.cc:1428] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
W20260929 03:08:30.387738 23904 tablet.cc:2369] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:08:30.411357 23770 tablet.cc:2369] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:08:30.432708 23636 tablet.cc:2369] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:08:30.453040 23927 raft_consensus.cc:482] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:08:30.453364 23927 raft_consensus.cc:504] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
I20260929 03:08:30.454960 23927 leader_election.cc:288] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239), 1e7d84ebe03848b3a53f547cd9890029 (127.7.209.65:33631)
I20260929 03:08:30.459429 23905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.455315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:33631 (local address 127.7.209.67:58125)
0929 03:08:30.455439 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:30.455456 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:30.455577 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:30.455889 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:30.455892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:30.455915 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:30.456049 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:30.456055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.457073 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.457077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:30.458224 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:30.458241 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.458971 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.458974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:30.459083 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.459110 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 03:08:30.459183 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 03:08:30.459270 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":1}
I20260929 03:08:30.459452 23918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.455448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:58125 (local address 127.7.209.65:33631)
0929 03:08:30.455587 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:30.455592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:30.455613 (+ 21us) server_negotiation.cc:407] Connection header received
0929 03:08:30.455696 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:30.455699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:30.455758 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:30.455847 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:30.456210 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.456947 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.458370 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.458856 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.458958 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.459010 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:30.459225 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 03:08:30.459313 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 03:08:30.459486 23929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.455315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:39239 (local address 127.7.209.67:35265)
0929 03:08:30.456126 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:30.456143 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:30.456256 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:30.456498 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:30.456500 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:30.456517 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:30.456648 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:30.456654 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.457521 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.457523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:30.458429 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:30.458439 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.459104 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.459106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:30.459199 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.459227 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 03:08:30.459299 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 03:08:30.459366 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"mutex_wait_us":354,"thread_start_us":171,"threads_started":1}
I20260929 03:08:30.459525 23915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.455560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:35265 (local address 127.7.209.66:39239)
0929 03:08:30.455710 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:30.455714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:30.456234 (+ 520us) server_negotiation.cc:407] Connection header received
0929 03:08:30.456361 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:30.456363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:30.456414 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:30.456471 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:30.456770 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.457399 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.458547 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.458991 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.459107 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.459158 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:30.459323 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 03:08:30.459403 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 03:08:30.460299 23589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5762effdd06443749a860fcf5622a933" candidate_uuid: "50ac64f16ffb4ae88ac445862599dafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7d84ebe03848b3a53f547cd9890029" is_pre_election: true
I20260929 03:08:30.460294 23723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5762effdd06443749a860fcf5622a933" candidate_uuid: "50ac64f16ffb4ae88ac445862599dafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b59d87795354e81adada2376afdd3fb" is_pre_election: true
I20260929 03:08:30.460628 23723 raft_consensus.cc:2454] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50ac64f16ffb4ae88ac445862599dafa in term 0.
I20260929 03:08:30.460628 23589 raft_consensus.cc:2454] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50ac64f16ffb4ae88ac445862599dafa in term 0.
I20260929 03:08:30.461122 23791 leader_election.cc:302] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ac64f16ffb4ae88ac445862599dafa, 7b59d87795354e81adada2376afdd3fb; no voters:
I20260929 03:08:30.461403 23927 raft_consensus.cc:2781] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:08:30.461494 23927 raft_consensus.cc:482] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:08:30.461544 23927 raft_consensus.cc:3037] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:30.462749 23927 raft_consensus.cc:504] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
I20260929 03:08:30.463088 23927 leader_election.cc:288] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [CANDIDATE]: Term 1 election: Requested vote from peers 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239), 1e7d84ebe03848b3a53f547cd9890029 (127.7.209.65:33631)
I20260929 03:08:30.463467 23723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5762effdd06443749a860fcf5622a933" candidate_uuid: "50ac64f16ffb4ae88ac445862599dafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b59d87795354e81adada2376afdd3fb"
I20260929 03:08:30.463469 23589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5762effdd06443749a860fcf5622a933" candidate_uuid: "50ac64f16ffb4ae88ac445862599dafa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e7d84ebe03848b3a53f547cd9890029"
I20260929 03:08:30.463594 23723 raft_consensus.cc:3037] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:30.463586 23589 raft_consensus.cc:3037] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:30.464704 23723 raft_consensus.cc:2454] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50ac64f16ffb4ae88ac445862599dafa in term 1.
I20260929 03:08:30.464763 23589 raft_consensus.cc:2454] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50ac64f16ffb4ae88ac445862599dafa in term 1.
I20260929 03:08:30.464987 23791 leader_election.cc:302] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50ac64f16ffb4ae88ac445862599dafa, 7b59d87795354e81adada2376afdd3fb; no voters:
I20260929 03:08:30.465188 23927 raft_consensus.cc:2781] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:08:30.465412 23927 raft_consensus.cc:686] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 1 LEADER]: Becoming Leader. State: Replica: 50ac64f16ffb4ae88ac445862599dafa, State: Running, Role: LEADER
I20260929 03:08:30.465816 23927 consensus_queue.cc:237] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
I20260929 03:08:30.469400 23442 catalog_manager.cc:5495] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa reported cstate change: term changed from 0 to 1, leader changed from <none> to 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67). New cstate: current_term: 1 leader_uuid: "50ac64f16ffb4ae88ac445862599dafa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } health_report { overall_health: HEALTHY } } }
I20260929 03:08:30.496311 23919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.492566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44170 (local address 127.7.209.67:33065)
0929 03:08:30.492710 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:30.492714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:30.492729 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:08:30.492836 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:30.492838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:30.492905 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:30.493003 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:08:30.493218 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.493830 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.494750 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:30.494983 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:30.495062 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:30.495328 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:08:30.495914 (+ 586us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:08:30.496020 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:30.496044 (+ 24us) server_negotiation.cc:299] Negotiation successful
0929 03:08:30.496107 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 03:08:30.503958 23723 raft_consensus.cc:1261] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 1 FOLLOWER]: Refusing update from remote peer 50ac64f16ffb4ae88ac445862599dafa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:08:30.503958 23589 raft_consensus.cc:1261] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Refusing update from remote peer 50ac64f16ffb4ae88ac445862599dafa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:08:30.504689 23927 consensus_queue.cc:1035] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:08:30.504905 23930 consensus_queue.cc:1035] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:08:30.514065 23934 mvcc.cc:205] Tried to move back new op lower bound from 7334507767815168000 to 7334507767671009280. Current Snapshot: MvccSnapshot[applied={T|T < 7334507767815168000}]
I20260929 03:08:30.522612 23443 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:08:30.522812 23443 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:08:30.525553 23443 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Sending DeleteTablet for 3 replicas of tablet 5762effdd06443749a860fcf5622a933
I20260929 03:08:30.526558 23569 tablet_service.cc:1514] Processing DeleteTablet for tablet 5762effdd06443749a860fcf5622a933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:30 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:54202
I20260929 03:08:30.526641 23703 tablet_service.cc:1514] Processing DeleteTablet for tablet 5762effdd06443749a860fcf5622a933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:30 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:36960
I20260929 03:08:30.527143 23943 tablet_replica.cc:330] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: stopping tablet replica
I20260929 03:08:30.527222 23837 tablet_service.cc:1514] Processing DeleteTablet for tablet 5762effdd06443749a860fcf5622a933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:30 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:44168
I20260929 03:08:30.527489 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 23506
I20260929 03:08:30.527597 23943 raft_consensus.cc:2229] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:08:30.528409 23945 tablet_replica.cc:330] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa: stopping tablet replica
I20260929 03:08:30.528721 23945 raft_consensus.cc:2229] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:08:30.529073 23944 tablet_replica.cc:330] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: stopping tablet replica
I20260929 03:08:30.529198 23945 raft_consensus.cc:2258] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:30.529383 23944 raft_consensus.cc:2229] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:08:30.529784 23944 raft_consensus.cc:2258] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:30.531459 23945 ts_tablet_manager.cc:1905] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:08:30.533967 23945 ts_tablet_manager.cc:1918] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:08:30.534055 23945 log.cc:1196] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/5762effdd06443749a860fcf5622a933
I20260929 03:08:30.534363 23945 ts_tablet_manager.cc:1939] T 5762effdd06443749a860fcf5622a933 P 50ac64f16ffb4ae88ac445862599dafa: Deleting consensus metadata
I20260929 03:08:30.535727 23429 catalog_manager.cc:4841] TS 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065): tablet 5762effdd06443749a860fcf5622a933 (table test-table [id=bd0f0516eabd4fe486e4e20acaefc0aa]) successfully deleted
W20260929 03:08:30.535946 23431 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:33631 (error 108)
I20260929 03:08:30.537009 23916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.536697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:33631 (local address 127.0.0.1:54216)
0929 03:08:30.536838 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:30.536917 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260929 03:08:30.537137 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 23640
W20260929 03:08:30.539413 23431 catalog_manager.cc:4584] TS 1e7d84ebe03848b3a53f547cd9890029 (127.7.209.65:33631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5762effdd06443749a860fcf5622a933: Network error: Client connection negotiation failed: client connection to 127.7.209.65:33631: connect: Connection refused (error 111)
I20260929 03:08:30.542936 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 23774
I20260929 03:08:30.544382 23916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:30.542977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:39239 (local address 127.0.0.1:36976)
0929 03:08:30.544234 (+ 1257us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:30.544298 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.66:39239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1160}
I20260929 03:08:30.548319 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 23412
I20260929 03:08:50.554509 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:44437
--webserver_interface=127.7.209.126
--webserver_port=33709
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:44437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:08:50.684859 23947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:50.685235 23947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:50.685281 23947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:50.689713 23947 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 03:08:50.689810 23947 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 03:08:50.689826 23947 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 03:08:50.689839 23947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:08:50.689859 23947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:50.689890 23947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:50.689924 23947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:08:50.689951 23947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:08:50.696075 23947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:44437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:44437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=33709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:50.697736 23947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:50.699051 23947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:50.714183 23953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:50.714303 23954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:50.714392 23947 server_base.cc:1034] running on GCE node
W20260929 03:08:50.714212 23956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:50.715200 23947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:50.716552 23947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:50.717670 23947 hybrid_clock.cc:648] HybridClock initialized: now 1790651330717632 us; error 53 us; skew 500 ppm
Sep 29 03:08:50 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:50.725095 23947 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:08:50.726361 23947 webserver.cc:460] Webserver started at http://127.7.209.126:33709/ using document root <none> and password file <none>
I20260929 03:08:50.727103 23947 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:50.727206 23947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:50.732012 23947 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 03:08:50.734508 23963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:50.736001 23947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:50.736135 23947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "a2165666b19148f2b38182ada4a15483"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "a2571d246946e08b5124f66990c7d828"
server_key_iv: "49e6817a6e0d6ade62deab758cd052fb"
server_key_version: "encryptionkey@0"
I20260929 03:08:50.736820 23947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:50.780510 23947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:50.784219 23947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:50.784435 23947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:50.792711 23947 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:44437
I20260929 03:08:50.793023 24015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:44437 every 8 connection(s)
I20260929 03:08:50.794145 23947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:08:50.799862 24016 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:08:50.801824 24016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Bootstrap starting.
I20260929 03:08:50.803340 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 23947
I20260929 03:08:50.803912 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:33631
--local_ip_for_outbound_sockets=127.7.209.65
--tserver_master_addrs=127.7.209.126:44437
--webserver_port=42903
--webserver_interface=127.7.209.65
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:08:50.822872 24016 log.cc:824] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:50.832721 24016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:08:50.833097 24016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Bootstrap complete.
I20260929 03:08:50.837350 24016 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } }
I20260929 03:08:50.837812 24016 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2165666b19148f2b38182ada4a15483, State: Initialized, Role: FOLLOWER
I20260929 03:08:50.838361 24016 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } }
I20260929 03:08:50.838493 24016 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:08:50.838570 24016 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:08:50.838672 24016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 1 FOLLOWER]: Advancing to term 2
I20260929 03:08:50.840832 24016 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } }
I20260929 03:08:50.841173 24016 leader_election.cc:302] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2165666b19148f2b38182ada4a15483; no voters:
I20260929 03:08:50.841504 24016 leader_election.cc:288] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 03:08:50.841602 24020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 03:08:50.841851 24020 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 2 LEADER]: Becoming Leader. State: Replica: a2165666b19148f2b38182ada4a15483, State: Running, Role: LEADER
I20260929 03:08:50.842162 24020 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } }
I20260929 03:08:50.842922 24016 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:08:50.843683 24021 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a2165666b19148f2b38182ada4a15483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } } }
I20260929 03:08:50.843811 24021 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:50.844385 24022 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2165666b19148f2b38182ada4a15483. Latest consensus state: current_term: 2 leader_uuid: "a2165666b19148f2b38182ada4a15483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } } }
I20260929 03:08:50.844478 24022 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: This master's current role is: LEADER
I20260929 03:08:50.845168 24026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:08:50.850631 24026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:08:50.851266 24026 catalog_manager.cc:1259] Loaded cluster ID: 2a0a08c511d440c08ecd16f81f520fc4
I20260929 03:08:50.851328 24026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:08:50.854084 24026 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:08:50.854809 24026 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Loaded TSK: 0
I20260929 03:08:50.870740 24026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Generated new TSK 1
I20260929 03:08:50.870994 24026 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 03:08:50.938488 24018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:50.938812 24018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:50.938861 24018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:50.943162 24018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:50.943236 24018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:50.943318 24018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:08:50.948866 24018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:33631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=42903
--tserver_master_addrs=127.7.209.126:44437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:50.950109 24018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:50.951292 24018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:50.962146 24044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:50.962132 24043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:50.962428 24018 server_base.cc:1034] running on GCE node
W20260929 03:08:50.962442 24046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:50.962914 24018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:50.963725 24018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:50.965777 24018 hybrid_clock.cc:648] HybridClock initialized: now 1790651330965739 us; error 53 us; skew 500 ppm
Sep 29 03:08:50 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:50.969857 24018 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:08:50.971038 24018 webserver.cc:460] Webserver started at http://127.7.209.65:42903/ using document root <none> and password file <none>
I20260929 03:08:50.971747 24018 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:50.971843 24018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:50.976612 24018 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260929 03:08:50.979283 24053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:50.980818 24018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:08:50.980935 24018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "1e7d84ebe03848b3a53f547cd9890029"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "f50a216f29e5dca3b7f41f6d8209a0e7"
server_key_iv: "43350602b7230a4614fda72b90557deb"
server_key_version: "encryptionkey@0"
I20260929 03:08:50.981564 24018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:51.017345 24018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:51.021392 24018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:51.021646 24018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:51.022451 24018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:51.023730 24060 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 03:08:51.028772 24018 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 03:08:51.028841 24018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260929 03:08:51.028910 24018 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 03:08:51.031006 24018 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 03:08:51.031054 24018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260929 03:08:51.031361 24060 tablet_bootstrap.cc:492] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Bootstrap starting.
I20260929 03:08:51.042618 24018 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:33631
I20260929 03:08:51.042680 24167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:33631 every 8 connection(s)
I20260929 03:08:51.043835 24018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:08:51.053158 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 24018
I20260929 03:08:51.053401 24060 log.cc:824] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:51.053820 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:39239
--local_ip_for_outbound_sockets=127.7.209.66
--tserver_master_addrs=127.7.209.126:44437
--webserver_port=40915
--webserver_interface=127.7.209.66
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 03:08:51 dist-test-slave-bdps krb5kdc[23399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:51.063383 24060 tablet_bootstrap.cc:492] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:08:51.063844 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.046388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:48431 (local address 127.7.209.126:44437)
0929 03:08:51.046831 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:51.046852 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:51.047398 (+ 546us) server_negotiation.cc:407] Connection header received
0929 03:08:51.048718 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:51.048730 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:51.049090 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:51.049454 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:51.050820 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.051940 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.053194 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.053858 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.054027 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.057580 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:51.057588 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:51.057641 (+ 53us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:51.057682 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:51.059853 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.060518 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.060524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.060529 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.060693 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.061084 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.061087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.061089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.061569 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:51.061733 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:51.062422 (+ 689us) server_negotiation.cc:299] Negotiation successful
0929 03:08:51.062744 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":126,"threads_started":1}
I20260929 03:08:51.063853 24171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.046590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:48431)
0929 03:08:51.047191 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:51.047241 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:51.048222 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:51.049528 (+ 1306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:51.049537 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:51.050066 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:51.050636 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.050653 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.052081 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.052084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.053074 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.053084 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.054244 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.054248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.054468 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.055264 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:51.055286 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:51.057349 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:51.059982 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:51.059989 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:51.060004 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:51.060392 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:51.060795 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:51.060798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:51.060799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:51.060985 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:51.061715 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:51.061723 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:51.062065 (+ 342us) client_negotiation.cc:769] Sending connection context
0929 03:08:51.062302 (+ 237us) client_negotiation.cc:240] Negotiation successful
0929 03:08:51.062691 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":165,"threads_started":1}
I20260929 03:08:51.063867 24060 tablet_bootstrap.cc:492] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Bootstrap complete.
I20260929 03:08:51.065310 24060 ts_tablet_manager.cc:1397] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Time spent bootstrapping tablet: real 0.034s user 0.024s sys 0.008s
I20260929 03:08:51.066856 24168 heartbeater.cc:347] Connected to a master server at 127.7.209.126:44437
I20260929 03:08:51.067127 24168 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:51.067811 24168 heartbeater.cc:511] Master 127.7.209.126:44437 requested a full tablet report, sending...
I20260929 03:08:51.068387 24060 raft_consensus.cc:348] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
I20260929 03:08:51.068948 24060 raft_consensus.cc:729] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7d84ebe03848b3a53f547cd9890029, State: Initialized, Role: FOLLOWER
I20260929 03:08:51.069481 24060 consensus_queue.cc:260] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
I20260929 03:08:51.070534 24060 ts_tablet_manager.cc:1428] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 03:08:51.071141 23980 ts_manager.cc:194] Registered new tserver with Master: 1e7d84ebe03848b3a53f547cd9890029 (127.7.209.65:33631)
I20260929 03:08:51.075878 23980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:48431
I20260929 03:08:51.078812 24168 heartbeater.cc:503] Master 127.7.209.126:44437 was elected leader, sending a full tablet report...
Sep 29 03:08:51 dist-test-slave-bdps krb5kdc[23399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.65@KRBTEST.COM
I20260929 03:08:51.087258 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.075172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:33631 (local address 127.0.0.1:44558)
0929 03:08:51.076043 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:51.076111 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:51.076295 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:51.076921 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:51.076927 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:51.077424 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:51.077573 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.077583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.079016 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.079020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.079929 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.079941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.080685 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.080688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.080784 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.081404 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:51.081417 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:51.083209 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:51.085352 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:51.085361 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:51.085366 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:51.085766 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:51.086063 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:51.086065 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:51.086066 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:51.086148 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:51.086653 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:51.086680 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:51.086796 (+ 116us) client_negotiation.cc:769] Sending connection context
0929 03:08:51.086911 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 03:08:51.087093 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":165,"threads_started":1}
I20260929 03:08:51.087402 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.075276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.7.209.65:33631)
0929 03:08:51.075680 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:51.075693 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:51.076217 (+ 524us) server_negotiation.cc:407] Connection header received
0929 03:08:51.076458 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:51.076463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:51.076640 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:51.076786 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:51.077729 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.078879 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.080058 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.080569 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.080680 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.083359 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:51.083369 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:51.083383 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:51.083409 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:51.085236 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.085870 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.085876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.085878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.085980 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.086227 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.086230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.086232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.086557 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:51.086691 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:51.087024 (+ 333us) server_negotiation.cc:299] Negotiation successful
0929 03:08:51.087223 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":96,"threads_started":1}
I20260929 03:08:51.088557 24102 tablet_service.cc:1514] Processing DeleteTablet for tablet 5762effdd06443749a860fcf5622a933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:30 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:44558
I20260929 03:08:51.089658 24177 tablet_replica.cc:330] stopping tablet replica
I20260929 03:08:51.090031 24177 raft_consensus.cc:2229] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:08:51.090345 24177 raft_consensus.cc:2258] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:51.091609 24177 ts_tablet_manager.cc:1905] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:08:51.095777 24177 ts_tablet_manager.cc:1918] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:08:51.095866 24177 log.cc:1196] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal/wals/5762effdd06443749a860fcf5622a933
I20260929 03:08:51.096104 24177 ts_tablet_manager.cc:1939] T 5762effdd06443749a860fcf5622a933 P 1e7d84ebe03848b3a53f547cd9890029: Deleting consensus metadata
I20260929 03:08:51.098208 23967 catalog_manager.cc:4841] TS 1e7d84ebe03848b3a53f547cd9890029 (127.7.209.65:33631): tablet 5762effdd06443749a860fcf5622a933 (table test-table [id=bd0f0516eabd4fe486e4e20acaefc0aa]) successfully deleted
W20260929 03:08:51.190301 24173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:51.190613 24173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:51.190677 24173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:51.194850 24173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:51.194926 24173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:51.195006 24173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:08:51.200423 24173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:39239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=40915
--tserver_master_addrs=127.7.209.126:44437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:51.202015 24173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:51.203567 24173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:51.216907 24183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:51.216924 24182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:51.216989 24173 server_base.cc:1034] running on GCE node
W20260929 03:08:51.216924 24185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:51.217715 24173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:51.218569 24173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:51.219723 24173 hybrid_clock.cc:648] HybridClock initialized: now 1790651331219691 us; error 42 us; skew 500 ppm
Sep 29 03:08:51 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:51.222782 24173 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:08:51.223770 24173 webserver.cc:460] Webserver started at http://127.7.209.66:40915/ using document root <none> and password file <none>
I20260929 03:08:51.224407 24173 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:51.224483 24173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:51.228967 24173 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:08:51.231410 24192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:51.232774 24173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:08:51.232879 24173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "7b59d87795354e81adada2376afdd3fb"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "a20dcb298b767972495ffcd197105449"
server_key_iv: "cc02bc3c439a7679ea1249b53d49b199"
server_key_version: "encryptionkey@0"
I20260929 03:08:51.233494 24173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:51.277004 24173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:51.280795 24173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:51.281003 24173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:51.281766 24173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:51.283108 24199 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 03:08:51.288582 24173 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 03:08:51.288651 24173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260929 03:08:51.288712 24173 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 03:08:51.290817 24173 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 03:08:51.290865 24173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260929 03:08:51.291214 24199 tablet_bootstrap.cc:492] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Bootstrap starting.
I20260929 03:08:51.298475 24199 log.cc:824] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:51.301555 24173 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:39239
I20260929 03:08:51.301662 24307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:39239 every 8 connection(s)
I20260929 03:08:51.302558 24173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:08:51.303268 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 24173
I20260929 03:08:51.303818 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:33065
--local_ip_for_outbound_sockets=127.7.209.67
--tserver_master_addrs=127.7.209.126:44437
--webserver_port=33741
--webserver_interface=127.7.209.67
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:08:51.309163 24199 tablet_bootstrap.cc:492] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:08:51.309559 24199 tablet_bootstrap.cc:492] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Bootstrap complete.
I20260929 03:08:51.309918 24199 ts_tablet_manager.cc:1397] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Time spent bootstrapping tablet: real 0.019s user 0.017s sys 0.000s
I20260929 03:08:51.313062 24199 raft_consensus.cc:348] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
I20260929 03:08:51.313575 24199 raft_consensus.cc:729] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b59d87795354e81adada2376afdd3fb, State: Initialized, Role: FOLLOWER
I20260929 03:08:51.314188 24199 consensus_queue.cc:260] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } }
Sep 29 03:08:51 dist-test-slave-bdps krb5kdc[23399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:51.315258 24199 ts_tablet_manager.cc:1428] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260929 03:08:51.319471 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.304974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:38227 (local address 127.7.209.126:44437)
0929 03:08:51.305163 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:51.305168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:51.305758 (+ 590us) server_negotiation.cc:407] Connection header received
0929 03:08:51.307184 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:51.307188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:51.307256 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:51.307365 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:51.308943 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.309865 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.311599 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.311977 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.312097 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.315749 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:51.315753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:51.315756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:51.315793 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:51.317394 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.317917 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.317921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.317923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.318020 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.318310 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.318313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.318315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.318550 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:51.318649 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:51.319123 (+ 474us) server_negotiation.cc:299] Negotiation successful
0929 03:08:51.319283 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 03:08:51.320125 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.304957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:38227)
0929 03:08:51.305519 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:51.305578 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:51.306897 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:51.307539 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:51.307548 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:51.308058 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:51.308756 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.308776 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.310078 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.310081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.311465 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.311475 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.312122 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.312126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.312335 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.313500 (+ 1165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:51.313521 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:51.315560 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:51.317502 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:51.317507 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:51.317518 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:51.317832 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:51.318097 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:51.318099 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:51.318100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:51.318228 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:51.318632 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:51.318636 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:51.318887 (+ 251us) client_negotiation.cc:769] Sending connection context
0929 03:08:51.319038 (+ 151us) client_negotiation.cc:240] Negotiation successful
0929 03:08:51.319305 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":145,"threads_started":1}
I20260929 03:08:51.321319 24308 heartbeater.cc:347] Connected to a master server at 127.7.209.126:44437
I20260929 03:08:51.321591 24308 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:51.322213 24308 heartbeater.cc:511] Master 127.7.209.126:44437 requested a full tablet report, sending...
I20260929 03:08:51.324343 23980 ts_manager.cc:194] Registered new tserver with Master: 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239)
I20260929 03:08:51.325676 23980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:38227
I20260929 03:08:51.328331 24308 heartbeater.cc:503] Master 127.7.209.126:44437 was elected leader, sending a full tablet report...
Sep 29 03:08:51 dist-test-slave-bdps krb5kdc[23399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.66@KRBTEST.COM
I20260929 03:08:51.335641 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.325341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:39239 (local address 127.0.0.1:54404)
0929 03:08:51.325464 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:51.325480 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:51.325559 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:51.326991 (+ 1432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:51.326994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:51.327204 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:51.327319 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.327326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.328301 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.328305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.329106 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.329116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.329933 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.329936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.330038 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.330590 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:51.330602 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:51.332086 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:51.334131 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:51.334134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:51.334136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:51.334354 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:51.334651 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:51.334654 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:51.334655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:51.334746 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:51.335205 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:51.335207 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:51.335292 (+ 85us) client_negotiation.cc:769] Sending connection context
0929 03:08:51.335368 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 03:08:51.335500 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 03:08:51.335983 24313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.325471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54404 (local address 127.7.209.66:39239)
0929 03:08:51.325811 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:51.325823 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:51.325847 (+ 24us) server_negotiation.cc:407] Connection header received
0929 03:08:51.325917 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:51.325923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:51.326151 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:51.327017 (+ 866us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:51.327421 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.328193 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.329228 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.329784 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.329930 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.332223 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:51.332233 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:51.332248 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:51.332273 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:51.334022 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.334457 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.334464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.334465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.334559 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.334821 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.334824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.334826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.335094 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:51.335228 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:51.335466 (+ 238us) server_negotiation.cc:299] Negotiation successful
0929 03:08:51.335621 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":142,"threads_started":1}
I20260929 03:08:51.337100 24241 tablet_service.cc:1514] Processing DeleteTablet for tablet 5762effdd06443749a860fcf5622a933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:30 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:54404
I20260929 03:08:51.338244 24314 tablet_replica.cc:330] stopping tablet replica
I20260929 03:08:51.338608 24314 raft_consensus.cc:2229] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:08:51.338915 24314 raft_consensus.cc:2258] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:51.340246 24314 ts_tablet_manager.cc:1905] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:08:51.345376 24314 ts_tablet_manager.cc:1918] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:08:51.345484 24314 log.cc:1196] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/5762effdd06443749a860fcf5622a933
I20260929 03:08:51.345903 24314 ts_tablet_manager.cc:1939] T 5762effdd06443749a860fcf5622a933 P 7b59d87795354e81adada2376afdd3fb: Deleting consensus metadata
I20260929 03:08:51.347855 23965 catalog_manager.cc:4841] TS 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239): tablet 5762effdd06443749a860fcf5622a933 (table test-table [id=bd0f0516eabd4fe486e4e20acaefc0aa]) successfully deleted
W20260929 03:08:51.438911 24311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:08:51.439179 24311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:08:51.439222 24311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:08:51.443341 24311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:08:51.443403 24311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:08:51.443470 24311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:08:51.448894 24311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:33065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=33741
--tserver_master_addrs=127.7.209.126:44437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:08:51.450238 24311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:08:51.451479 24311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:08:51.462186 24319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:51.462261 24311 server_base.cc:1034] running on GCE node
W20260929 03:08:51.462208 24320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:08:51.462267 24322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:08:51.462793 24311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:08:51.463480 24311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:08:51.464644 24311 hybrid_clock.cc:648] HybridClock initialized: now 1790651331464622 us; error 35 us; skew 500 ppm
Sep 29 03:08:51 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:08:51.467720 24311 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:08:51.468724 24311 webserver.cc:460] Webserver started at http://127.7.209.67:33741/ using document root <none> and password file <none>
I20260929 03:08:51.469323 24311 fs_manager.cc:359] Metadata directory not provided
I20260929 03:08:51.469400 24311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:08:51.473868 24311 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:08:51.476221 24329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:51.477646 24311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:08:51.477775 24311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "50ac64f16ffb4ae88ac445862599dafa"
format_stamp: "Formatted at 2026-09-29 03:08:30 on dist-test-slave-bdps"
server_key: "bedb3696135ca42472ac86cb9f68416e"
server_key_iv: "31dfd8c02cb999a8f273c7c7811069cb"
server_key_version: "encryptionkey@0"
I20260929 03:08:51.478456 24311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:08:51.498924 24311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:08:51.502496 24311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:08:51.502701 24311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:08:51.503455 24311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:08:51.504590 24311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:08:51.504654 24311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:51.504714 24311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:08:51.504750 24311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:08:51.515601 24311 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:33065
I20260929 03:08:51.515625 24442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:33065 every 8 connection(s)
I20260929 03:08:51.516901 24311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:08:51.522172 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 24311
Sep 29 03:08:51 dist-test-slave-bdps krb5kdc[23399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:08:51.531394 24447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.522973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57734 (local address 127.7.209.126:44437)
0929 03:08:51.523334 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:51.523339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:51.523383 (+ 44us) server_negotiation.cc:407] Connection header received
0929 03:08:51.523489 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:51.523492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:51.523539 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:51.523637 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:51.524427 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.524975 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.526018 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.526245 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.526315 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.527809 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:51.527813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:51.527816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:51.527846 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:51.529522 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.530069 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.530073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.530075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.530182 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.530465 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.530467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.530469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.530655 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:51.530841 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:51.531052 (+ 211us) server_negotiation.cc:299] Negotiation successful
0929 03:08:51.531185 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":82,"threads_started":1}
I20260929 03:08:51.533031 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.519104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:48465 (local address 127.7.209.126:44437)
0929 03:08:51.519301 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:51.519306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:51.520544 (+ 1238us) server_negotiation.cc:407] Connection header received
0929 03:08:51.521531 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:51.521534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:51.521590 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:51.521709 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:51.523354 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.524042 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.525486 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.525758 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.525838 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.529187 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:51.529197 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:51.529199 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:51.529239 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:51.530785 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.531377 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.531380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.531381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.531490 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.531879 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.531882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.531883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.532078 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:51.532155 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:51.532778 (+ 623us) server_negotiation.cc:299] Negotiation successful
0929 03:08:51.532888 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 03:08:51.534221 24445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.519614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:48465)
0929 03:08:51.520378 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:51.520423 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:51.521332 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:51.521912 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:51.521923 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:51.522484 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:08:51.523189 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.523209 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.524169 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.524173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.525363 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.525373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.525908 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.525912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.526112 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.526776 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:08:51.526800 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:08:51.529010 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:08:51.530900 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:51.530907 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:51.530928 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:51.531292 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:51.531602 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:08:51.531605 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:08:51.531607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:08:51.531788 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:08:51.532187 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:08:51.532203 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:08:51.532534 (+ 331us) client_negotiation.cc:769] Sending connection context
0929 03:08:51.532724 (+ 190us) client_negotiation.cc:240] Negotiation successful
0929 03:08:51.533119 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":138,"threads_started":1}
I20260929 03:08:51.535318 24443 heartbeater.cc:347] Connected to a master server at 127.7.209.126:44437
I20260929 03:08:51.535561 24443 heartbeater.cc:464] Registering TS with master...
I20260929 03:08:51.536092 24443 heartbeater.cc:511] Master 127.7.209.126:44437 requested a full tablet report, sending...
I20260929 03:08:51.537204 23980 ts_manager.cc:194] Registered new tserver with Master: 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065)
I20260929 03:08:51.537889 23980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:48465
I20260929 03:08:51.545271 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:08:51.549211 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.545944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.7.209.126:44437)
0929 03:08:51.546060 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:51.546064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:51.546274 (+ 210us) server_negotiation.cc:407] Connection header received
0929 03:08:51.546412 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:51.546415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:51.546524 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:51.546612 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:08:51.546827 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.547347 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.548166 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.548385 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.548461 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.548726 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:08:51.548885 (+ 159us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0929 03:08:51.549101 (+ 216us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57750: token expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260929 03:08:51.549270 24170 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57750: token expired
W20260929 03:08:51.549651 23909 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.209.126:44437)
I20260929 03:08:51.556051 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.550030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57764 (local address 127.7.209.126:44437)
0929 03:08:51.550130 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:51.550134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:51.550146 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:08:51.550217 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:51.550219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:51.550254 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:51.550316 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:08:51.550886 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.551351 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.552074 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.552280 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.552359 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.553644 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:08:51.553649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:08:51.553652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:08:51.553675 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:08:51.554852 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.555202 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.555204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.555205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.555272 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:08:51.555449 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:08:51.555450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:08:51.555451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:08:51.555602 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:08:51.555700 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:51.555842 (+ 142us) server_negotiation.cc:299] Negotiation successful
0929 03:08:51.555930 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 03:08:51.558735 23980 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:08:51.562865 23980 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:08:51.572093 24102 tablet_service.cc:1467] Processing CreateTablet for tablet ff5c8fc75aed4c61b611b8a765bf9ae6 (DEFAULT_TABLE table=test-table [id=809912fb648b473cac658551c7995136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:51.572137 24241 tablet_service.cc:1467] Processing CreateTablet for tablet ff5c8fc75aed4c61b611b8a765bf9ae6 (DEFAULT_TABLE table=test-table [id=809912fb648b473cac658551c7995136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:51.572954 24102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff5c8fc75aed4c61b611b8a765bf9ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:51.572954 24241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff5c8fc75aed4c61b611b8a765bf9ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:51.576481 24199 tablet_bootstrap.cc:492] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb: Bootstrap starting.
I20260929 03:08:51.576699 24453 tablet_bootstrap.cc:492] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029: Bootstrap starting.
I20260929 03:08:51.577735 24453 tablet_bootstrap.cc:654] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:51.577736 24199 tablet_bootstrap.cc:654] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:51.579196 24453 tablet_bootstrap.cc:492] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029: No bootstrap required, opened a new log
I20260929 03:08:51.579345 24199 tablet_bootstrap.cc:492] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb: No bootstrap required, opened a new log
I20260929 03:08:51.579438 24199 ts_tablet_manager.cc:1397] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 03:08:51.579440 24453 ts_tablet_manager.cc:1397] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260929 03:08:51.579893 24199 raft_consensus.cc:348] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:08:51.579893 24453 raft_consensus.cc:348] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:08:51.580025 24199 raft_consensus.cc:374] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:51.580025 24453 raft_consensus.cc:374] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:51.580068 24453 raft_consensus.cc:729] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7d84ebe03848b3a53f547cd9890029, State: Initialized, Role: FOLLOWER
I20260929 03:08:51.580068 24199 raft_consensus.cc:729] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b59d87795354e81adada2376afdd3fb, State: Initialized, Role: FOLLOWER
I20260929 03:08:51.580214 24199 consensus_queue.cc:260] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:08:51.580246 24453 consensus_queue.cc:260] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:08:51.580538 24199 ts_tablet_manager.cc:1428] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 03:08:51.580590 24453 ts_tablet_manager.cc:1428] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260929 03:08:51.581602 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.571203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:33065 (local address 127.0.0.1:49640)
0929 03:08:51.571326 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:51.571344 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:51.571434 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:51.577392 (+ 5958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:51.577396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:51.577416 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:51.577555 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.577562 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.578889 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.578893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.579973 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.579984 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.581084 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.581088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.581196 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.581256 (+ 60us) client_negotiation.cc:769] Sending connection context
0929 03:08:51.581344 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 03:08:51.581452 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260929 03:08:51.582010 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.571782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49640 (local address 127.7.209.67:33065)
0929 03:08:51.576813 (+ 5031us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:51.576829 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:51.576863 (+ 34us) server_negotiation.cc:407] Connection header received
0929 03:08:51.576968 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:51.576973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:51.577219 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:51.577406 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:51.577747 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.578744 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.580129 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.580963 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.581050 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.581201 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:51.581742 (+ 541us) server_negotiation.cc:299] Negotiation successful
0929 03:08:51.581859 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4883,"thread_start_us":134,"threads_started":1}
I20260929 03:08:51.585577 24377 tablet_service.cc:1467] Processing CreateTablet for tablet ff5c8fc75aed4c61b611b8a765bf9ae6 (DEFAULT_TABLE table=test-table [id=809912fb648b473cac658551c7995136]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:08:51.586817 24377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff5c8fc75aed4c61b611b8a765bf9ae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:08:51.594484 24457 tablet_bootstrap.cc:492] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa: Bootstrap starting.
I20260929 03:08:51.597090 24457 tablet_bootstrap.cc:654] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa: Neither blocks nor log segments found. Creating new log.
I20260929 03:08:51.597961 24457 log.cc:824] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa: Log is configured to *not* fsync() on all Append() calls
I20260929 03:08:51.600132 24457 tablet_bootstrap.cc:492] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa: No bootstrap required, opened a new log
I20260929 03:08:51.600368 24457 ts_tablet_manager.cc:1397] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:08:51.603698 24457 raft_consensus.cc:348] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:08:51.603971 24457 raft_consensus.cc:374] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:08:51.604022 24457 raft_consensus.cc:729] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50ac64f16ffb4ae88ac445862599dafa, State: Initialized, Role: FOLLOWER
I20260929 03:08:51.604532 24457 consensus_queue.cc:260] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:08:51.605302 24443 heartbeater.cc:503] Master 127.7.209.126:44437 was elected leader, sending a full tablet report...
I20260929 03:08:51.605749 24457 ts_tablet_manager.cc:1428] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260929 03:08:51.767151 24456 raft_consensus.cc:482] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:08:51.767423 24456 raft_consensus.cc:504] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
W20260929 03:08:51.767997 24444 tablet.cc:2369] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:08:51.768895 24456 leader_election.cc:288] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065), 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239)
I20260929 03:08:51.774114 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.769297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:57303 (local address 127.7.209.67:33065)
0929 03:08:51.769473 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:51.769479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:51.769590 (+ 111us) server_negotiation.cc:407] Connection header received
0929 03:08:51.769877 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:51.769881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:51.769940 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:51.770016 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:51.770299 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.771133 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.772782 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.773389 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.773519 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.773581 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:51.773895 (+ 314us) server_negotiation.cc:299] Negotiation successful
0929 03:08:51.773974 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 03:08:51.774114 24313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.769797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:55699 (local address 127.7.209.66:39239)
0929 03:08:51.769941 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:51.769946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:51.769963 (+ 17us) server_negotiation.cc:407] Connection header received
0929 03:08:51.770118 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:51.770122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:51.770181 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:51.770280 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:08:51.770517 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.771272 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.772773 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.773471 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.773573 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.773760 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:51.773873 (+ 113us) server_negotiation.cc:299] Negotiation successful
0929 03:08:51.773961 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 03:08:51.774134 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.769178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:33065 (local address 127.7.209.65:57303)
0929 03:08:51.769487 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:51.769502 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:51.769682 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:51.770044 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:51.770048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:51.770063 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:51.770192 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.770197 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.771247 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.771250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.772643 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.772655 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.773521 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.773526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.773670 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.773783 (+ 113us) client_negotiation.cc:769] Sending connection context
0929 03:08:51.773852 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 03:08:51.773962 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":138,"thread_start_us":100,"threads_started":1}
I20260929 03:08:51.774137 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.769646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:39239 (local address 127.7.209.65:55699)
0929 03:08:51.769902 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:51.769913 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 03:08:51.770006 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:08:51.770304 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:08:51.770307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:08:51.770323 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:08:51.770424 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.770428 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.771361 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.771364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.772643 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:08:51.772655 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.773570 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.773573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:08:51.773684 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.773778 (+ 94us) client_negotiation.cc:769] Sending connection context
0929 03:08:51.773858 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 03:08:51.773962 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":30,"thread_start_us":67,"threads_started":1}
I20260929 03:08:51.774856 24397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff5c8fc75aed4c61b611b8a765bf9ae6" candidate_uuid: "1e7d84ebe03848b3a53f547cd9890029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ac64f16ffb4ae88ac445862599dafa" is_pre_election: true
I20260929 03:08:51.774930 24261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff5c8fc75aed4c61b611b8a765bf9ae6" candidate_uuid: "1e7d84ebe03848b3a53f547cd9890029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b59d87795354e81adada2376afdd3fb" is_pre_election: true
I20260929 03:08:51.775234 24397 raft_consensus.cc:2454] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e7d84ebe03848b3a53f547cd9890029 in term 0.
I20260929 03:08:51.775243 24261 raft_consensus.cc:2454] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e7d84ebe03848b3a53f547cd9890029 in term 0.
I20260929 03:08:51.775688 24055 leader_election.cc:302] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7d84ebe03848b3a53f547cd9890029, 50ac64f16ffb4ae88ac445862599dafa; no voters:
I20260929 03:08:51.775947 24456 raft_consensus.cc:2781] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:08:51.776037 24456 raft_consensus.cc:482] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:08:51.776093 24456 raft_consensus.cc:3037] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:51.777272 24456 raft_consensus.cc:504] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:08:51.777594 24456 leader_election.cc:288] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 1 election: Requested vote from peers 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065), 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239)
I20260929 03:08:51.778018 24261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff5c8fc75aed4c61b611b8a765bf9ae6" candidate_uuid: "1e7d84ebe03848b3a53f547cd9890029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b59d87795354e81adada2376afdd3fb"
I20260929 03:08:51.778017 24397 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff5c8fc75aed4c61b611b8a765bf9ae6" candidate_uuid: "1e7d84ebe03848b3a53f547cd9890029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50ac64f16ffb4ae88ac445862599dafa"
I20260929 03:08:51.778154 24261 raft_consensus.cc:3037] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:51.778154 24397 raft_consensus.cc:3037] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:08:51.779610 24397 raft_consensus.cc:2454] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e7d84ebe03848b3a53f547cd9890029 in term 1.
I20260929 03:08:51.779610 24261 raft_consensus.cc:2454] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e7d84ebe03848b3a53f547cd9890029 in term 1.
I20260929 03:08:51.779884 24055 leader_election.cc:302] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e7d84ebe03848b3a53f547cd9890029, 7b59d87795354e81adada2376afdd3fb; no voters:
I20260929 03:08:51.780030 24456 raft_consensus.cc:2781] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:08:51.780231 24456 raft_consensus.cc:686] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 LEADER]: Becoming Leader. State: Replica: 1e7d84ebe03848b3a53f547cd9890029, State: Running, Role: LEADER
I20260929 03:08:51.780551 24456 consensus_queue.cc:237] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:08:51.784137 23979 catalog_manager.cc:5495] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e7d84ebe03848b3a53f547cd9890029 (127.7.209.65). New cstate: current_term: 1 leader_uuid: "1e7d84ebe03848b3a53f547cd9890029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } health_report { overall_health: UNKNOWN } } }
W20260929 03:08:51.795576 24169 tablet.cc:2369] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:08:51.804121 24309 tablet.cc:2369] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:08:51.815181 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.810981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44572 (local address 127.7.209.65:33631)
0929 03:08:51.811512 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0929 03:08:51.811519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:08:51.811534 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:08:51.811634 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:08:51.811636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:08:51.811700 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:08:51.811841 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:08:51.812092 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.812645 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.813465 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:08:51.813713 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:08:51.813798 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:08:51.814016 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:08:51.814572 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:08:51.814661 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 03:08:51.814689 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 03:08:51.814761 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":332,"threads_started":1}
I20260929 03:08:51.819918 24397 raft_consensus.cc:1261] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa [term 1 FOLLOWER]: Refusing update from remote peer 1e7d84ebe03848b3a53f547cd9890029: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:08:51.819913 24261 raft_consensus.cc:1261] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb [term 1 FOLLOWER]: Refusing update from remote peer 1e7d84ebe03848b3a53f547cd9890029: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:08:51.820676 24462 consensus_queue.cc:1035] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:08:51.820883 24456 consensus_queue.cc:1035] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:08:51.826496 24466 mvcc.cc:205] Tried to move back new op lower bound from 7334507855127044096 to 7334507854975774720. Current Snapshot: MvccSnapshot[applied={T|T < 7334507855127044096}]
I20260929 03:08:51.831007 24468 mvcc.cc:205] Tried to move back new op lower bound from 7334507855127044096 to 7334507854975774720. Current Snapshot: MvccSnapshot[applied={T|T < 7334507855127044096}]
I20260929 03:08:51.835570 23980 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:08:51.835762 23980 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:08:51.838382 23980 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Sending DeleteTablet for 3 replicas of tablet ff5c8fc75aed4c61b611b8a765bf9ae6
I20260929 03:08:51.839066 24377 tablet_service.cc:1514] Processing DeleteTablet for tablet ff5c8fc75aed4c61b611b8a765bf9ae6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:51 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:49640
I20260929 03:08:51.839592 24476 tablet_replica.cc:330] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa: stopping tablet replica
I20260929 03:08:51.839920 24241 tablet_service.cc:1514] Processing DeleteTablet for tablet ff5c8fc75aed4c61b611b8a765bf9ae6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:51 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:54404
I20260929 03:08:51.839944 24102 tablet_service.cc:1514] Processing DeleteTablet for tablet ff5c8fc75aed4c61b611b8a765bf9ae6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:08:51 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:44558
I20260929 03:08:51.839996 24476 raft_consensus.cc:2229] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:08:51.840104 24314 tablet_replica.cc:330] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb: stopping tablet replica
I20260929 03:08:51.840230 24314 raft_consensus.cc:2229] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:08:51.840338 24314 raft_consensus.cc:2258] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:51.840340 24476 raft_consensus.cc:2258] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:08:51.840962 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 24018
I20260929 03:08:51.841184 24314 ts_tablet_manager.cc:1905] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:08:51.843768 24314 ts_tablet_manager.cc:1918] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:08:51.843843 24314 log.cc:1196] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/ff5c8fc75aed4c61b611b8a765bf9ae6
I20260929 03:08:51.844240 24314 ts_tablet_manager.cc:1939] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 7b59d87795354e81adada2376afdd3fb: Deleting consensus metadata
I20260929 03:08:51.844316 24476 ts_tablet_manager.cc:1905] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:08:51.844945 23965 catalog_manager.cc:4841] TS 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239): tablet ff5c8fc75aed4c61b611b8a765bf9ae6 (table test-table [id=809912fb648b473cac658551c7995136]) successfully deleted
I20260929 03:08:51.847287 24476 ts_tablet_manager.cc:1918] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:08:51.847388 24476 log.cc:1196] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/ff5c8fc75aed4c61b611b8a765bf9ae6
I20260929 03:08:51.847731 24476 ts_tablet_manager.cc:1939] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 50ac64f16ffb4ae88ac445862599dafa: Deleting consensus metadata
I20260929 03:08:51.849071 23965 catalog_manager.cc:4841] TS 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065): tablet ff5c8fc75aed4c61b611b8a765bf9ae6 (table test-table [id=809912fb648b473cac658551c7995136]) successfully deleted
W20260929 03:08:51.850330 23967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:33631 (error 108)
I20260929 03:08:51.851289 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 24173
I20260929 03:08:51.851444 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:08:51.851160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:33631 (local address 127.0.0.1:44574)
0929 03:08:51.851286 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 03:08:51.851351 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 03:08:51.852041 23967 catalog_manager.cc:4584] TS 1e7d84ebe03848b3a53f547cd9890029 (127.7.209.65:33631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ff5c8fc75aed4c61b611b8a765bf9ae6: Network error: Client connection negotiation failed: client connection to 127.7.209.65:33631: connect: Connection refused (error 111)
I20260929 03:08:51.857091 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 24311
I20260929 03:08:51.862488 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 23947
I20260929 03:09:11.868921 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:44437
--webserver_interface=127.7.209.126
--webserver_port=33709
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:44437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:11.991989 24479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:11.992292 24479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:11.992352 24479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:11.996420 24479 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 03:09:11.996471 24479 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 03:09:11.996492 24479 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 03:09:11.996510 24479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:11.996532 24479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:11.996562 24479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:11.996592 24479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:11.996619 24479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:12.001938 24479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:44437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:44437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=33709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:12.003151 24479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:12.004374 24479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:12.013933 24487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:12.013918 24484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:12.013939 24479 server_base.cc:1034] running on GCE node
W20260929 03:09:12.013962 24485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:12.014788 24479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:12.016032 24479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:12.017203 24479 hybrid_clock.cc:648] HybridClock initialized: now 1790651352017186 us; error 35 us; skew 500 ppm
Sep 29 03:09:12 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651352, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:12.020291 24479 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:09:12.021304 24479 webserver.cc:460] Webserver started at http://127.7.209.126:33709/ using document root <none> and password file <none>
I20260929 03:09:12.021997 24479 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:12.022073 24479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:12.026533 24479 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260929 03:09:12.028919 24494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:12.030517 24479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:12.030643 24479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "a2165666b19148f2b38182ada4a15483"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "a2571d246946e08b5124f66990c7d828"
server_key_iv: "49e6817a6e0d6ade62deab758cd052fb"
server_key_version: "encryptionkey@0"
I20260929 03:09:12.031246 24479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:12.067005 24479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:12.070434 24479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:12.070638 24479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:12.078583 24479 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:44437
I20260929 03:09:12.078979 24546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:44437 every 8 connection(s)
I20260929 03:09:12.079851 24479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:12.085150 24547 sys_catalog.cc:263] Verifying existing consensus state
I20260929 03:09:12.087045 24547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Bootstrap starting.
I20260929 03:09:12.087045 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 24479
I20260929 03:09:12.087522 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:33631
--local_ip_for_outbound_sockets=127.7.209.65
--tserver_master_addrs=127.7.209.126:44437
--webserver_port=42903
--webserver_interface=127.7.209.65
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:09:12.112221 24547 log.cc:824] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:12.127334 24547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:09:12.127802 24547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Bootstrap complete.
I20260929 03:09:12.133496 24547 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } }
I20260929 03:09:12.134050 24547 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2165666b19148f2b38182ada4a15483, State: Initialized, Role: FOLLOWER
I20260929 03:09:12.134675 24547 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } }
I20260929 03:09:12.134850 24547 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:12.134917 24547 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:12.135013 24547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 2 FOLLOWER]: Advancing to term 3
I20260929 03:09:12.137647 24547 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } }
I20260929 03:09:12.138023 24547 leader_election.cc:302] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2165666b19148f2b38182ada4a15483; no voters:
I20260929 03:09:12.138368 24547 leader_election.cc:288] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 03:09:12.138473 24551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 03:09:12.138689 24551 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [term 3 LEADER]: Becoming Leader. State: Replica: a2165666b19148f2b38182ada4a15483, State: Running, Role: LEADER
I20260929 03:09:12.139019 24551 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } }
I20260929 03:09:12.139796 24547 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:12.140566 24552 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2165666b19148f2b38182ada4a15483. Latest consensus state: current_term: 3 leader_uuid: "a2165666b19148f2b38182ada4a15483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } } }
I20260929 03:09:12.140571 24553 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a2165666b19148f2b38182ada4a15483" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2165666b19148f2b38182ada4a15483" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 44437 } } }
I20260929 03:09:12.140841 24553 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:12.140866 24552 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:12.141464 24560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:12.147369 24560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:12.151022 24560 catalog_manager.cc:1259] Loaded cluster ID: 2a0a08c511d440c08ecd16f81f520fc4
I20260929 03:09:12.151131 24560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:12.154729 24560 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:12.155578 24560 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Loaded TSK: 0
I20260929 03:09:12.155721 24560 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a2165666b19148f2b38182ada4a15483: Loaded TSK: 1
F20260929 03:09:12.156558 24560 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f421df869ad google::LogMessage::Fail() at ??:0
@ 0x7f421df8988a google::LogMessage::SendToLog() at ??:0
@ 0x7f421df864de google::LogMessage::Flush() at ??:0
@ 0x7f421df87a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4220c4a18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f4220c42f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f42288eb7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f42288c599c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f42288c6178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f4228907df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f422892682b std::function<>::operator()() at ??:0
@ 0x7f42288c5b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f42288c6f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f42288c3579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f4228907640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f42201d83d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f42201d8c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f42201da3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f42201c9512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f422180d6db start_thread at ??:0
@ 0x7f421ca8461f clone at ??:0
W20260929 03:09:12.251763 24549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:12.251991 24549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:12.252030 24549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:12.256167 24549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:12.256227 24549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:12.256304 24549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:09:12.261582 24549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:33631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=42903
--tserver_master_addrs=127.7.209.126:44437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:12.262857 24549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:12.264047 24549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:12.274749 24594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:12.274838 24549 server_base.cc:1034] running on GCE node
W20260929 03:09:12.274991 24597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:12.275091 24595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:12.275527 24549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:12.276189 24549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:12.277361 24549 hybrid_clock.cc:648] HybridClock initialized: now 1790651352277338 us; error 35 us; skew 500 ppm
Sep 29 03:09:12 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651352, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:12.280157 24549 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:09:12.281039 24549 webserver.cc:460] Webserver started at http://127.7.209.65:42903/ using document root <none> and password file <none>
I20260929 03:09:12.281646 24549 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:12.281721 24549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:12.285925 24549 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:09:12.288268 24604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:12.289799 24549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:12.289902 24549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "1e7d84ebe03848b3a53f547cd9890029"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "f50a216f29e5dca3b7f41f6d8209a0e7"
server_key_iv: "43350602b7230a4614fda72b90557deb"
server_key_version: "encryptionkey@0"
I20260929 03:09:12.290503 24549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:12.305848 24549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:12.309365 24549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:12.309569 24549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:12.310280 24549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:12.311509 24611 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260929 03:09:12.317870 24549 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 03:09:12.317981 24549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260929 03:09:12.318073 24549 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 03:09:12.319965 24549 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 03:09:12.320010 24549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260929 03:09:12.320386 24611 tablet_bootstrap.cc:492] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029: Bootstrap starting.
I20260929 03:09:12.329830 24549 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:33631
I20260929 03:09:12.330065 24718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:33631 every 8 connection(s)
I20260929 03:09:12.330897 24549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:09:12.337242 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 24549
I20260929 03:09:12.337838 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:39239
--local_ip_for_outbound_sockets=127.7.209.66
--tserver_master_addrs=127.7.209.126:44437
--webserver_port=40915
--webserver_interface=127.7.209.66
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:09:12.346899 24611 log.cc:824] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:12.359252 24611 tablet_bootstrap.cc:492] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 03:09:12.359826 24611 tablet_bootstrap.cc:492] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029: Bootstrap complete.
I20260929 03:09:12.361126 24611 ts_tablet_manager.cc:1397] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029: Time spent bootstrapping tablet: real 0.041s user 0.025s sys 0.012s
I20260929 03:09:12.363780 24611 raft_consensus.cc:348] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:09:12.364213 24611 raft_consensus.cc:729] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e7d84ebe03848b3a53f547cd9890029, State: Initialized, Role: FOLLOWER
I20260929 03:09:12.364921 24611 consensus_queue.cc:260] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:09:12.366180 24611 ts_tablet_manager.cc:1428] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260929 03:09:12.415585 24721 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:09:12.332995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:44235)
0929 03:09:12.333527 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:12.333586 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:12.334497 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:12.414696 (+ 80199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":140,"threads_started":1}
W20260929 03:09:12.416225 24719 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:44437 (0 consecutive failures): Network error: Failed to ping master at 127.7.209.126:44437: Client connection negotiation failed: client connection to 127.7.209.126:44437: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 03:09:12.417207 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:12.416900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:57409)
0929 03:09:12.417049 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:12.417135 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 03:09:12.418051 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:12.417797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:55197)
0929 03:09:12.417925 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:12.417984 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 03:09:12.418237 24719 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 03:09:12.485884 24722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:12.486150 24722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:12.486192 24722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:12.490181 24722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:12.490228 24722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:12.490294 24722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:09:12.495375 24722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:39239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=40915
--tserver_master_addrs=127.7.209.126:44437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:12.496502 24722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:12.497728 24722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:12.508157 24732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:12.508167 24730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:12.508167 24729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:12.508420 24722 server_base.cc:1034] running on GCE node
I20260929 03:09:12.508872 24722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:12.509526 24722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:12.510679 24722 hybrid_clock.cc:648] HybridClock initialized: now 1790651352510647 us; error 45 us; skew 500 ppm
Sep 29 03:09:12 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651352, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:12.513803 24722 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:09:12.514755 24722 webserver.cc:460] Webserver started at http://127.7.209.66:40915/ using document root <none> and password file <none>
I20260929 03:09:12.515370 24722 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:12.515440 24722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:12.519680 24722 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:09:12.521842 24739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:12.523101 24722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 03:09:12.523206 24722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "7b59d87795354e81adada2376afdd3fb"
format_stamp: "Formatted at 2026-09-29 03:08:29 on dist-test-slave-bdps"
server_key: "a20dcb298b767972495ffcd197105449"
server_key_iv: "cc02bc3c439a7679ea1249b53d49b199"
server_key_version: "encryptionkey@0"
I20260929 03:09:12.523785 24722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:12.535445 24722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:12.538273 24722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:12.538452 24722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:12.539077 24722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:12.539966 24722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:12.540016 24722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:12.540071 24722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:12.540098 24722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:12.549928 24722 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:39239
I20260929 03:09:12.549950 24852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:39239 every 8 connection(s)
I20260929 03:09:12.551020 24722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:09:12.555028 24855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:12.553362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:43669)
0929 03:09:12.554096 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:12.554215 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":139,"threads_started":1}
W20260929 03:09:12.555466 24853 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:44437 (0 consecutive failures): Network error: Failed to ping master at 127.7.209.126:44437: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
I20260929 03:09:12.556085 24855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:12.555856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:33739)
0929 03:09:12.555983 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:12.556034 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 03:09:12.556631 24855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:12.556454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:56853)
0929 03:09:12.556540 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:12.556582 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 03:09:12.556785 24853 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 03:09:12.558290 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 24722
I20260929 03:09:12.558748 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:33065
--local_ip_for_outbound_sockets=127.7.209.67
--tserver_master_addrs=127.7.209.126:44437
--webserver_port=33741
--webserver_interface=127.7.209.67
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:12.682042 24856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:12.682327 24856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:12.682370 24856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:12.686403 24856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:12.686458 24856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:12.686527 24856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:09:12.691744 24856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:33065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=33741
--tserver_master_addrs=127.7.209.126:44437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:12.692883 24856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:12.694067 24856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:12.704823 24864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:12.704823 24861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:12.705070 24856 server_base.cc:1034] running on GCE node
W20260929 03:09:12.705148 24862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:12.705564 24856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:12.706285 24856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:12.707435 24856 hybrid_clock.cc:648] HybridClock initialized: now 1790651352707408 us; error 46 us; skew 500 ppm
Sep 29 03:09:12 dist-test-slave-bdps krb5kdc[23399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651352, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:12.710338 24856 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:09:12.711256 24856 webserver.cc:460] Webserver started at http://127.7.209.67:33741/ using document root <none> and password file <none>
I20260929 03:09:12.711858 24856 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:12.711931 24856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:12.716130 24856 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 03:09:12.718427 24871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:12.719782 24856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:09:12.719888 24856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "50ac64f16ffb4ae88ac445862599dafa"
format_stamp: "Formatted at 2026-09-29 03:08:30 on dist-test-slave-bdps"
server_key: "bedb3696135ca42472ac86cb9f68416e"
server_key_iv: "31dfd8c02cb999a8f273c7c7811069cb"
server_key_version: "encryptionkey@0"
I20260929 03:09:12.720489 24856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:12.729756 24856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:12.732622 24856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:12.732791 24856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:12.733377 24856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:12.734270 24856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:12.734325 24856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:12.734381 24856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:12.734409 24856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:12.744210 24856 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:33065
I20260929 03:09:12.744225 24984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:33065 every 8 connection(s)
I20260929 03:09:12.745731 24856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:09:12.750365 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:12.748531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:49857)
0929 03:09:12.749315 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:12.749448 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":120,"threads_started":1}
W20260929 03:09:12.750867 24985 heartbeater.cc:644] Failed to heartbeat to 127.7.209.126:44437 (0 consecutive failures): Network error: Failed to ping master at 127.7.209.126:44437: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
I20260929 03:09:12.751492 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:12.751289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:35569)
0929 03:09:12.751387 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:12.751437 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260929 03:09:12.752133 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:12.751936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:45957)
0929 03:09:12.752027 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:12.752074 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260929 03:09:12.752296 24985 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 03:09:12.755849 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 24856
I20260929 03:09:13.419391 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:13.418892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:48757)
0929 03:09:13.419165 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:13.419261 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":91,"threads_started":1}
I20260929 03:09:13.557704 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:13.557294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:58967)
0929 03:09:13.557529 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:13.557580 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":81,"threads_started":1}
I20260929 03:09:13.640931 24991 raft_consensus.cc:482] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:09:13.641193 24991 raft_consensus.cc:504] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:09:13.642694 24991 leader_election.cc:288] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065), 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239)
Sep 29 03:09:13 dist-test-slave-bdps krb5kdc[23399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651352, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.66@KRBTEST.COM
Sep 29 03:09:13 dist-test-slave-bdps krb5kdc[23399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651352, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.67@KRBTEST.COM
I20260929 03:09:13.657295 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:13.643542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:44941 (local address 127.7.209.66:39239)
0929 03:09:13.643926 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:13.643948 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:13.643982 (+ 34us) server_negotiation.cc:407] Connection header received
0929 03:09:13.644666 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:13.644694 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:13.644990 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:13.645273 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:13.646673 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:13.648556 (+ 1883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:13.649344 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:13.649894 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:13.650052 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:13.652741 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:13.652748 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:13.652764 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:13.652798 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:13.654484 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:13.655092 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:13.655099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:13.655105 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:13.655280 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:13.655630 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:13.655633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:13.655635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:13.656120 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:13.656223 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:13.656825 (+ 602us) server_negotiation.cc:299] Negotiation successful
0929 03:09:13.657094 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":72,"threads_started":1}
I20260929 03:09:13.657279 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:13.642928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:33065 (local address 127.7.209.65:42065)
0929 03:09:13.643056 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:13.643074 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:13.643206 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:13.645306 (+ 2100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:13.645313 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:13.645917 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:13.646498 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:13.646509 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:13.648333 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:13.648336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:13.649231 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:13.649240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:13.650008 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:13.650011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:13.650201 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:13.650855 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:09:13.650871 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:13.652883 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:13.655007 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:13.655010 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:13.655013 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:13.655259 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:13.655630 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:13.655633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:13.655635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:13.655776 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:13.656408 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:13.656411 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:13.656502 (+ 91us) client_negotiation.cc:769] Sending connection context
0929 03:09:13.656691 (+ 189us) client_negotiation.cc:240] Negotiation successful
0929 03:09:13.657035 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":74,"spinlock_wait_cycles":252672}
I20260929 03:09:13.657301 24992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:13.643113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:42065 (local address 127.7.209.67:33065)
0929 03:09:13.643541 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:13.643556 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:13.643605 (+ 49us) server_negotiation.cc:407] Connection header received
0929 03:09:13.644415 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:13.644422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:13.644880 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:13.645180 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:13.646688 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:13.648191 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:13.649350 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:13.649895 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:13.650051 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:13.653072 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:13.653078 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:13.653088 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:13.653118 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:13.654882 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:13.655374 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:13.655381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:13.655386 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:13.655535 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:13.655864 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:13.655866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:13.655867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:13.656304 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:13.656433 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:13.656829 (+ 396us) server_negotiation.cc:299] Negotiation successful
0929 03:09:13.657095 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":92,"threads_started":1}
I20260929 03:09:13.657287 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:13.643240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:39239 (local address 127.7.209.65:44941)
0929 03:09:13.643522 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:13.643538 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:13.643669 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:13.645306 (+ 1637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:13.645313 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:13.645917 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:13.646482 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:13.646500 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:13.648678 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:13.648682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:13.649233 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:13.649241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:13.650009 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:13.650013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:13.650199 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:13.650890 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:09:13.650903 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:13.652511 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:13.654600 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:13.654606 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:13.654621 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:13.654969 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:13.655378 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:13.655381 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:13.655383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:13.655546 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:13.656237 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:13.656243 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:13.656505 (+ 262us) client_negotiation.cc:769] Sending connection context
0929 03:09:13.656692 (+ 187us) client_negotiation.cc:240] Negotiation successful
0929 03:09:13.657110 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":244,"spinlock_wait_cycles":4480,"thread_start_us":87,"threads_started":1}
W20260929 03:09:13.660787 24606 leader_election.cc:341] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065): Not found: Tablet not found: ff5c8fc75aed4c61b611b8a765bf9ae6
W20260929 03:09:13.661005 24606 leader_election.cc:341] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239): Not found: Tablet not found: ff5c8fc75aed4c61b611b8a765bf9ae6
I20260929 03:09:13.661067 24606 leader_election.cc:302] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e7d84ebe03848b3a53f547cd9890029; no voters: 50ac64f16ffb4ae88ac445862599dafa, 7b59d87795354e81adada2376afdd3fb
I20260929 03:09:13.661345 24991 raft_consensus.cc:2726] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 03:09:13.753265 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:13.752819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:38339)
0929 03:09:13.753084 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:13.753143 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":83,"threads_started":1}
I20260929 03:09:14.420735 24996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:14.420255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:50479)
0929 03:09:14.420551 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:14.420615 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":108,"threads_started":1}
I20260929 03:09:14.558895 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:14.558475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:53451)
0929 03:09:14.558730 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:14.558794 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":89,"threads_started":1}
I20260929 03:09:14.754683 24998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:14.754155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:33005)
0929 03:09:14.754475 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:14.754554 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":107,"threads_started":1}
I20260929 03:09:15.422456 24999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:15.421869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:42687)
0929 03:09:15.422248 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:15.422333 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":112,"threads_started":1}
I20260929 03:09:15.560211 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:15.559727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:33955)
0929 03:09:15.560022 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:15.560094 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":93,"threads_started":1}
I20260929 03:09:15.650790 25001 raft_consensus.cc:482] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:09:15.650971 25001 raft_consensus.cc:504] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:09:15.651481 25001 leader_election.cc:288] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065), 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239)
W20260929 03:09:15.652588 24606 leader_election.cc:341] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065): Not found: Tablet not found: ff5c8fc75aed4c61b611b8a765bf9ae6
W20260929 03:09:15.652732 24606 leader_election.cc:341] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239): Not found: Tablet not found: ff5c8fc75aed4c61b611b8a765bf9ae6
I20260929 03:09:15.652817 24606 leader_election.cc:302] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e7d84ebe03848b3a53f547cd9890029; no voters: 50ac64f16ffb4ae88ac445862599dafa, 7b59d87795354e81adada2376afdd3fb
I20260929 03:09:15.652982 25001 raft_consensus.cc:2726] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 03:09:15.756088 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:15.755595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:50253)
0929 03:09:15.755906 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:15.755982 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":114,"threads_started":1}
I20260929 03:09:16.424232 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:16.423591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:45533)
0929 03:09:16.424017 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:16.424103 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":132,"threads_started":1}
I20260929 03:09:16.561635 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:16.561114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:38709)
0929 03:09:16.561426 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:16.561500 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":98,"threads_started":1}
I20260929 03:09:16.757272 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:16.756802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:59089)
0929 03:09:16.757099 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:16.757155 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":113,"threads_started":1}
I20260929 03:09:17.425981 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:17.425385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:38931)
0929 03:09:17.425781 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:17.425857 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":93,"threads_started":1}
I20260929 03:09:17.562991 25007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:17.562489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:60167)
0929 03:09:17.562812 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:17.562878 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":118,"threads_started":1}
I20260929 03:09:17.758558 25008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:17.758059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:43221)
0929 03:09:17.758382 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:17.758444 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":121,"threads_started":1}
I20260929 03:09:17.969435 25009 raft_consensus.cc:482] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:09:17.969641 25009 raft_consensus.cc:504] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:09:17.970086 25009 leader_election.cc:288] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065), 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239)
W20260929 03:09:17.971009 24606 leader_election.cc:341] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065): Not found: Tablet not found: ff5c8fc75aed4c61b611b8a765bf9ae6
W20260929 03:09:17.971140 24606 leader_election.cc:341] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239): Not found: Tablet not found: ff5c8fc75aed4c61b611b8a765bf9ae6
I20260929 03:09:17.971194 24606 leader_election.cc:302] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e7d84ebe03848b3a53f547cd9890029; no voters: 50ac64f16ffb4ae88ac445862599dafa, 7b59d87795354e81adada2376afdd3fb
I20260929 03:09:17.971355 25009 raft_consensus.cc:2726] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 03:09:18.427294 25010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:18.426783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:45019)
0929 03:09:18.427099 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:18.427166 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":91,"threads_started":1}
I20260929 03:09:18.564191 25011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:18.563733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:51307)
0929 03:09:18.564030 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:18.564090 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":111,"threads_started":1}
I20260929 03:09:18.759763 25012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:18.759318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:55339)
0929 03:09:18.759588 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:18.759646 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
I20260929 03:09:19.428710 25013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:19.428169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:58167)
0929 03:09:19.428517 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:19.428586 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":111,"threads_started":1}
I20260929 03:09:19.565492 25014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:19.564949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:38607)
0929 03:09:19.565322 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:19.565383 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":114,"threads_started":1}
I20260929 03:09:19.760957 25015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:19.760477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:33755)
0929 03:09:19.760788 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:19.760849 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":85,"threads_started":1}
I20260929 03:09:20.430251 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:20.429692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:39469)
0929 03:09:20.430040 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:20.430114 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":136,"threads_started":1}
I20260929 03:09:20.529093 25017 raft_consensus.cc:482] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:09:20.529275 25017 raft_consensus.cc:504] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:09:20.529776 25017 leader_election.cc:288] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065), 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239)
W20260929 03:09:20.530604 24606 leader_election.cc:341] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239): Not found: Tablet not found: ff5c8fc75aed4c61b611b8a765bf9ae6
W20260929 03:09:20.530740 24606 leader_election.cc:341] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065): Not found: Tablet not found: ff5c8fc75aed4c61b611b8a765bf9ae6
I20260929 03:09:20.530889 24606 leader_election.cc:302] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e7d84ebe03848b3a53f547cd9890029; no voters: 50ac64f16ffb4ae88ac445862599dafa, 7b59d87795354e81adada2376afdd3fb
I20260929 03:09:20.531036 25017 raft_consensus.cc:2726] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 03:09:20.566676 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:20.566229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:35993)
0929 03:09:20.566504 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:20.566563 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":91,"threads_started":1}
I20260929 03:09:20.762202 25019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:20.761745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:56725)
0929 03:09:20.762033 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:20.762092 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":89,"threads_started":1}
I20260929 03:09:21.431568 25020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:21.431056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:50077)
0929 03:09:21.431380 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:21.431447 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":126,"threads_started":1}
I20260929 03:09:21.567929 25021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:21.567416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:46761)
0929 03:09:21.567726 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:21.567809 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":95,"threads_started":1}
I20260929 03:09:21.763347 25022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:21.762916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:40583)
0929 03:09:21.763178 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:21.763235 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":93,"threads_started":1}
I20260929 03:09:22.079025 25023 raft_consensus.cc:482] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:09:22.079217 25023 raft_consensus.cc:504] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50ac64f16ffb4ae88ac445862599dafa" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 33065 } } peers { permanent_uuid: "1e7d84ebe03848b3a53f547cd9890029" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33631 } } peers { permanent_uuid: "7b59d87795354e81adada2376afdd3fb" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39239 } }
I20260929 03:09:22.079694 25023 leader_election.cc:288] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065), 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239)
W20260929 03:09:22.080575 24606 leader_election.cc:341] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b59d87795354e81adada2376afdd3fb (127.7.209.66:39239): Not found: Tablet not found: ff5c8fc75aed4c61b611b8a765bf9ae6
W20260929 03:09:22.080708 24606 leader_election.cc:341] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 50ac64f16ffb4ae88ac445862599dafa (127.7.209.67:33065): Not found: Tablet not found: ff5c8fc75aed4c61b611b8a765bf9ae6
I20260929 03:09:22.080746 24606 leader_election.cc:302] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1e7d84ebe03848b3a53f547cd9890029; no voters: 50ac64f16ffb4ae88ac445862599dafa, 7b59d87795354e81adada2376afdd3fb
I20260929 03:09:22.080875 25023 raft_consensus.cc:2726] T ff5c8fc75aed4c61b611b8a765bf9ae6 P 1e7d84ebe03848b3a53f547cd9890029 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 03:09:22.433140 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:22.432643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:43789)
0929 03:09:22.432915 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:22.432980 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":88,"threads_started":1}
I20260929 03:09:22.569283 25025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:22.568778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:36051)
0929 03:09:22.569084 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:22.569154 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":98,"threads_started":1}
I20260929 03:09:22.764564 25026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:22.764112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:55159)
0929 03:09:22.764385 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:22.764449 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":92,"threads_started":1}
I20260929 03:09:23.434608 25027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:23.434096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:33549)
0929 03:09:23.434407 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:23.434482 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":94,"threads_started":1}
I20260929 03:09:23.570785 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:23.570266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:50539)
0929 03:09:23.570567 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:23.570643 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":100,"threads_started":1}
I20260929 03:09:23.765806 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:23.765347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:46289)
0929 03:09:23.765623 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:23.765692 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":90,"threads_started":1}
I20260929 03:09:24.436067 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:24.435525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:43501)
0929 03:09:24.435861 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:24.435940 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":104,"threads_started":1}
I20260929 03:09:24.572053 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:24.571598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:34241)
0929 03:09:24.571874 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:24.571935 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":90,"threads_started":1}
I20260929 03:09:24.767058 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:24.766558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:33699)
0929 03:09:24.766874 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:24.766936 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":122,"threads_started":1}
I20260929 03:09:25.437546 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:25.436972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:40143)
0929 03:09:25.437319 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:25.437418 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":106,"threads_started":1}
I20260929 03:09:25.573408 25034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:25.572923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:50101)
0929 03:09:25.573220 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:25.573288 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":100,"threads_started":1}
I20260929 03:09:25.768390 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:25.767907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:39617)
0929 03:09:25.768201 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:25.768266 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":99,"threads_started":1}
I20260929 03:09:26.438917 25036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:26.438420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:37745)
0929 03:09:26.438714 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:26.438783 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":97,"threads_started":1}
I20260929 03:09:26.574774 25037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:26.574285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:53757)
0929 03:09:26.574581 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:26.574647 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":95,"threads_started":1}
I20260929 03:09:26.769982 25038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:26.769369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.67:40241)
0929 03:09:26.769759 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:26.769850 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":105,"threads_started":1}
I20260929 03:09:27.440649 25039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:27.440141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.65:45963)
0929 03:09:27.440448 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:27.440521 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":111,"threads_started":1}
I20260929 03:09:27.576196 25040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:27.575696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:44437 (local address 127.7.209.66:52319)
0929 03:09:27.576000 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:27.576066 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.126:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":93,"threads_started":1}
I20260929 03:09:27.759002 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 24549
I20260929 03:09:27.765439 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 24722
I20260929 03:09:27.771149 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 24856
2026-09-29T03:09:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59718 ms)
[----------] 30 tests from SecurityITest (247275 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:27 dist-test-slave-bdps krb5kdc[25044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:27 dist-test-slave-bdps krb5kdc[25044](info): set up 2 sockets
Sep 29 03:09:27 dist-test-slave-bdps krb5kdc[25044](info): commencing operation
krb5kdc: starting...
W20260929 03:09:28.729790 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.007s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:28 dist-test-slave-bdps krb5kdc[25044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:09:28.852051 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:33763
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:33763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:28.980355 25057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:28.980675 25057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:28.980716 25057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:28.984781 25057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:28.984862 25057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:28.984882 25057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:28.984898 25057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:28.984915 25057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:28.990921 25057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:33763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:33763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:28.992328 25057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:28.993714 25057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:29.003928 25063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:29.003993 25057 server_base.cc:1034] running on GCE node
W20260929 03:09:29.003944 25062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:29.003944 25065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:29.004809 25057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:29.006183 25057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:29.007345 25057 hybrid_clock.cc:648] HybridClock initialized: now 1790651369007321 us; error 43 us; skew 500 ppm
Sep 29 03:09:29 dist-test-slave-bdps krb5kdc[25044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:29.010504 25057 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:09:29.011554 25057 webserver.cc:460] Webserver started at http://127.7.209.126:39583/ using document root <none> and password file <none>
I20260929 03:09:29.012190 25057 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:29.012267 25057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:29.012497 25057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:29.014942 25057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "471370f51fab4b78b41728ff3b965290"
format_stamp: "Formatted at 2026-09-29 03:09:29 on dist-test-slave-bdps"
server_key: "5b3dadec303ab2f55bd68acc046d2e9f"
server_key_iv: "6ea8e5711bfe25e4a1fcec460151d752"
server_key_version: "encryptionkey@0"
I20260929 03:09:29.015571 25057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "471370f51fab4b78b41728ff3b965290"
format_stamp: "Formatted at 2026-09-29 03:09:29 on dist-test-slave-bdps"
server_key: "5b3dadec303ab2f55bd68acc046d2e9f"
server_key_iv: "6ea8e5711bfe25e4a1fcec460151d752"
server_key_version: "encryptionkey@0"
I20260929 03:09:29.019829 25057 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 03:09:29.022814 25072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:29.024293 25057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:29.024418 25057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "471370f51fab4b78b41728ff3b965290"
format_stamp: "Formatted at 2026-09-29 03:09:29 on dist-test-slave-bdps"
server_key: "5b3dadec303ab2f55bd68acc046d2e9f"
server_key_iv: "6ea8e5711bfe25e4a1fcec460151d752"
server_key_version: "encryptionkey@0"
I20260929 03:09:29.024533 25057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:29.042790 25057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:29.046554 25057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:29.046757 25057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:29.054402 25057 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:33763
I20260929 03:09:29.054423 25124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:33763 every 8 connection(s)
I20260929 03:09:29.055466 25057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:29.058776 25125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:29.059659 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 25057
I20260929 03:09:29.059803 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:29.060185 8005 external_mini_cluster.cc:1428] Setting key 711787c61a1098df71fca0e62e4704b5
I20260929 03:09:29.067072 25125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290: Bootstrap starting.
I20260929 03:09:29.071149 25125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290: Neither blocks nor log segments found. Creating new log.
Sep 29 03:09:29 dist-test-slave-bdps krb5kdc[25044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:09:29.072216 25125 log.cc:824] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:29.074873 25125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290: No bootstrap required, opened a new log
I20260929 03:09:29.077117 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.061942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54928 (local address 127.7.209.126:33763)
0929 03:09:29.062408 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:29.062431 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:29.062476 (+ 45us) server_negotiation.cc:407] Connection header received
0929 03:09:29.063215 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:29.063247 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:29.063693 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:29.064064 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:29.065137 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.066451 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.067826 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.068425 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.068568 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.071963 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:29.071972 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:29.071994 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:29.072035 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:29.073936 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:29.074431 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:29.074436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:29.074441 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:29.074558 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:29.074861 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:29.074864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:29.074865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:29.075386 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:29.075544 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:29.075856 (+ 312us) server_negotiation.cc:299] Negotiation successful
0929 03:09:29.076201 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":125,"threads_started":1}
I20260929 03:09:29.079268 25125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471370f51fab4b78b41728ff3b965290" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 33763 } }
I20260929 03:09:29.079576 25125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:29.079639 25125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 471370f51fab4b78b41728ff3b965290, State: Initialized, Role: FOLLOWER
I20260929 03:09:29.080138 25125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471370f51fab4b78b41728ff3b965290" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 33763 } }
I20260929 03:09:29.080300 25125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:29.080366 25125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:29.080469 25125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:29.081482 25125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471370f51fab4b78b41728ff3b965290" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 33763 } }
I20260929 03:09:29.081878 25125 leader_election.cc:302] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 471370f51fab4b78b41728ff3b965290; no voters:
I20260929 03:09:29.082212 25125 leader_election.cc:288] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:29.082326 25130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:29.082541 25130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [term 1 LEADER]: Becoming Leader. State: Replica: 471370f51fab4b78b41728ff3b965290, State: Running, Role: LEADER
I20260929 03:09:29.082894 25130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471370f51fab4b78b41728ff3b965290" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 33763 } }
I20260929 03:09:29.083585 25125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:29.084646 25131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "471370f51fab4b78b41728ff3b965290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471370f51fab4b78b41728ff3b965290" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 33763 } } }
I20260929 03:09:29.084807 25131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:29.085179 25132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 471370f51fab4b78b41728ff3b965290. Latest consensus state: current_term: 1 leader_uuid: "471370f51fab4b78b41728ff3b965290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "471370f51fab4b78b41728ff3b965290" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 33763 } } }
I20260929 03:09:29.085933 25132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:29.085947 25139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:29.089017 25139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:29.094853 25139 catalog_manager.cc:1347] Generated new cluster ID: 53e5b06d783942d69bd1a95a71edb599
I20260929 03:09:29.094914 25139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:29.103701 25139 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:29.104637 25139 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:29.126396 25139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290: Generated new TSK 0
I20260929 03:09:29.127256 25139 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:09:29.158877 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:33763
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:29.286625 25151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:29.286943 25151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:29.286988 25151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:29.291188 25151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:29.291265 25151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:29.291335 25151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:09:29.296990 25151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:33763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:29.298444 25151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:29.299657 25151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:29.310838 25157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:29.310861 25159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:29.310856 25156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:29.311100 25151 server_base.cc:1034] running on GCE node
I20260929 03:09:29.311641 25151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:29.312414 25151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:29.313572 25151 hybrid_clock.cc:648] HybridClock initialized: now 1790651369313544 us; error 42 us; skew 500 ppm
Sep 29 03:09:29 dist-test-slave-bdps krb5kdc[25044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:29.316680 25151 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:09:29.317756 25151 webserver.cc:460] Webserver started at http://127.7.209.65:33187/ using document root <none> and password file <none>
I20260929 03:09:29.318413 25151 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:29.318475 25151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:29.318679 25151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:29.321040 25151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "99a1725ac07d4fec8b55b026383dc36a"
format_stamp: "Formatted at 2026-09-29 03:09:29 on dist-test-slave-bdps"
server_key: "383a8dd7eed541f5175f87f59ac0707a"
server_key_iv: "b6b8484c08dcb79581491aa56074ca61"
server_key_version: "encryptionkey@0"
I20260929 03:09:29.321666 25151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "99a1725ac07d4fec8b55b026383dc36a"
format_stamp: "Formatted at 2026-09-29 03:09:29 on dist-test-slave-bdps"
server_key: "383a8dd7eed541f5175f87f59ac0707a"
server_key_iv: "b6b8484c08dcb79581491aa56074ca61"
server_key_version: "encryptionkey@0"
I20260929 03:09:29.325964 25151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:09:29.328630 25166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:29.330109 25151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:29.330221 25151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "99a1725ac07d4fec8b55b026383dc36a"
format_stamp: "Formatted at 2026-09-29 03:09:29 on dist-test-slave-bdps"
server_key: "383a8dd7eed541f5175f87f59ac0707a"
server_key_iv: "b6b8484c08dcb79581491aa56074ca61"
server_key_version: "encryptionkey@0"
I20260929 03:09:29.330328 25151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:29.340142 25151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:29.343091 25151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:29.343261 25151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:29.343852 25151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:29.344717 25151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:29.344762 25151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:29.344805 25151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:29.344834 25151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:29.354436 25151 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:38971
I20260929 03:09:29.354460 25279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:38971 every 8 connection(s)
I20260929 03:09:29.355641 25151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:09:29.357525 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 25151
I20260929 03:09:29.357692 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:09:29.358114 8005 external_mini_cluster.cc:1428] Setting key 1210a7fdc4ff6bdf3d75addfb0ea5a50
Sep 29 03:09:29 dist-test-slave-bdps krb5kdc[25044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
I20260929 03:09:29.371858 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.358021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:54027 (local address 127.7.209.126:33763)
0929 03:09:29.358210 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:29.358216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:29.359420 (+ 1204us) server_negotiation.cc:407] Connection header received
0929 03:09:29.360991 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:29.360994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:29.361055 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:29.361150 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:29.362764 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.363358 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.364991 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.365222 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.365297 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.368296 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:29.368301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:29.368303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:29.368328 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:29.369737 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:29.370212 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:29.370216 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:29.370218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:29.370301 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:29.370646 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:29.370649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:29.370651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:29.370853 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:29.370940 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:29.371520 (+ 580us) server_negotiation.cc:299] Negotiation successful
0929 03:09:29.371660 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 03:09:29.372810 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.358574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:33763 (local address 127.7.209.65:54027)
0929 03:09:29.359222 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:29.359292 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:29.360747 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:29.361340 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:29.361349 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:29.361819 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:29.362588 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:29.362610 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.363488 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.363492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:29.364888 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:29.364899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.365391 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.365396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:29.365650 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.366352 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:09:29.366375 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:29.368128 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:29.369844 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:29.369850 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:29.369867 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:29.370120 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:29.370382 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:29.370385 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:29.370387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:29.370561 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:29.370950 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:29.370957 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:29.371218 (+ 261us) client_negotiation.cc:769] Sending connection context
0929 03:09:29.371466 (+ 248us) client_negotiation.cc:240] Negotiation successful
0929 03:09:29.371846 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":187,"threads_started":1}
I20260929 03:09:29.374135 25280 heartbeater.cc:347] Connected to a master server at 127.7.209.126:33763
I20260929 03:09:29.374456 25280 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:29.375026 25280 heartbeater.cc:511] Master 127.7.209.126:33763 requested a full tablet report, sending...
I20260929 03:09:29.376912 25089 ts_manager.cc:194] Registered new tserver with Master: 99a1725ac07d4fec8b55b026383dc36a (127.7.209.65:38971)
I20260929 03:09:29.378724 25089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:54027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:09:29.388285 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:33763
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:29.514249 25285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:29.514556 25285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:29.514603 25285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:29.518740 25285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:29.518797 25285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:29.518857 25285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:09:29.524418 25285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:33763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:29.525580 25285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:29.526859 25285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:29.537317 25291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:29.537319 25290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:29.537492 25285 server_base.cc:1034] running on GCE node
W20260929 03:09:29.537690 25293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:29.538084 25285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:29.538738 25285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:29.539891 25285 hybrid_clock.cc:648] HybridClock initialized: now 1790651369539873 us; error 30 us; skew 500 ppm
Sep 29 03:09:29 dist-test-slave-bdps krb5kdc[25044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:29.542747 25285 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:09:29.543660 25285 webserver.cc:460] Webserver started at http://127.7.209.66:37179/ using document root <none> and password file <none>
I20260929 03:09:29.544267 25285 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:29.544335 25285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:29.544579 25285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:29.546907 25285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "aff882ff2aa147c5bacf957f6a25cefd"
format_stamp: "Formatted at 2026-09-29 03:09:29 on dist-test-slave-bdps"
server_key: "0daddd8ddd73b53ed59f0db5b300b0b3"
server_key_iv: "4cb6b0081d3374eb70e89d32f4ae1647"
server_key_version: "encryptionkey@0"
I20260929 03:09:29.547519 25285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "aff882ff2aa147c5bacf957f6a25cefd"
format_stamp: "Formatted at 2026-09-29 03:09:29 on dist-test-slave-bdps"
server_key: "0daddd8ddd73b53ed59f0db5b300b0b3"
server_key_iv: "4cb6b0081d3374eb70e89d32f4ae1647"
server_key_version: "encryptionkey@0"
I20260929 03:09:29.551726 25285 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:09:29.554383 25300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:29.555790 25285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:09:29.555902 25285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "aff882ff2aa147c5bacf957f6a25cefd"
format_stamp: "Formatted at 2026-09-29 03:09:29 on dist-test-slave-bdps"
server_key: "0daddd8ddd73b53ed59f0db5b300b0b3"
server_key_iv: "4cb6b0081d3374eb70e89d32f4ae1647"
server_key_version: "encryptionkey@0"
I20260929 03:09:29.555997 25285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:29.569463 25285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:29.572345 25285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:29.572520 25285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:29.573114 25285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:29.574004 25285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:29.574052 25285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:29.574105 25285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:29.574133 25285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:29.583549 25285 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:37303
I20260929 03:09:29.583570 25413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:37303 every 8 connection(s)
I20260929 03:09:29.584693 25285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:09:29.585294 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 25285
I20260929 03:09:29.585433 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:09:29.585877 8005 external_mini_cluster.cc:1428] Setting key 2787f7a7f7599f14ffb5279f992a9a99
Sep 29 03:09:29 dist-test-slave-bdps krb5kdc[25044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
I20260929 03:09:29.600594 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.586990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:58139 (local address 127.7.209.126:33763)
0929 03:09:29.587139 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:29.587144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:29.588555 (+ 1411us) server_negotiation.cc:407] Connection header received
0929 03:09:29.589876 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:29.589879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:29.589936 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:29.590010 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:29.591548 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.592128 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.593475 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.593760 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.593878 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.596955 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:29.596958 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:29.596960 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:29.596989 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:29.598409 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:29.598897 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:29.598901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:29.598903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:29.598990 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:29.599308 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:29.599312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:29.599314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:29.599552 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:29.599649 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:29.600219 (+ 570us) server_negotiation.cc:299] Negotiation successful
0929 03:09:29.600375 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 03:09:29.601550 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.587589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:33763 (local address 127.7.209.66:58139)
0929 03:09:29.588374 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:29.588439 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:29.589591 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:29.590254 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:29.590266 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:29.590806 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:29.591394 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:29.591409 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.592253 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.592257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:29.593356 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:29.593365 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.593972 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.593977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:29.594215 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.594890 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:09:29.594914 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:29.596774 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:29.598511 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:29.598517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:29.598534 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:29.598803 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:29.599072 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:29.599075 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:29.599078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:29.599220 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:29.599655 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:29.599662 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:29.599957 (+ 295us) client_negotiation.cc:769] Sending connection context
0929 03:09:29.600155 (+ 198us) client_negotiation.cc:240] Negotiation successful
0929 03:09:29.600524 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":143,"threads_started":1}
I20260929 03:09:29.602892 25414 heartbeater.cc:347] Connected to a master server at 127.7.209.126:33763
I20260929 03:09:29.603214 25414 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:29.603890 25414 heartbeater.cc:511] Master 127.7.209.126:33763 requested a full tablet report, sending...
I20260929 03:09:29.605218 25089 ts_manager.cc:194] Registered new tserver with Master: aff882ff2aa147c5bacf957f6a25cefd (127.7.209.66:37303)
I20260929 03:09:29.605908 25089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:58139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:09:29.615792 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:33763
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:29.737892 25419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:29.738148 25419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:29.738189 25419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:29.742193 25419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:29.742256 25419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:29.742318 25419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:09:29.747731 25419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:33763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:29.749131 25419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:29.750473 25419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:29.761739 25425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:29.761729 25424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:29.761729 25427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:29.761798 25419 server_base.cc:1034] running on GCE node
I20260929 03:09:29.762591 25419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:29.763355 25419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:29.764514 25419 hybrid_clock.cc:648] HybridClock initialized: now 1790651369764494 us; error 37 us; skew 500 ppm
Sep 29 03:09:29 dist-test-slave-bdps krb5kdc[25044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:29.767784 25419 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:09:29.768891 25419 webserver.cc:460] Webserver started at http://127.7.209.67:40209/ using document root <none> and password file <none>
I20260929 03:09:29.769573 25419 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:29.769695 25419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:29.769964 25419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:29.772521 25419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "5da5e4a0bc0444a3b9fd23e9610e57af"
format_stamp: "Formatted at 2026-09-29 03:09:29 on dist-test-slave-bdps"
server_key: "0837f275a7d4872793052d29713ae0fa"
server_key_iv: "8a0e886981f16d87ed3ae19c7c5f1406"
server_key_version: "encryptionkey@0"
I20260929 03:09:29.773188 25419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "5da5e4a0bc0444a3b9fd23e9610e57af"
format_stamp: "Formatted at 2026-09-29 03:09:29 on dist-test-slave-bdps"
server_key: "0837f275a7d4872793052d29713ae0fa"
server_key_iv: "8a0e886981f16d87ed3ae19c7c5f1406"
server_key_version: "encryptionkey@0"
I20260929 03:09:29.777565 25419 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 03:09:29.780313 25434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:29.781687 25419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:29.781817 25419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "5da5e4a0bc0444a3b9fd23e9610e57af"
format_stamp: "Formatted at 2026-09-29 03:09:29 on dist-test-slave-bdps"
server_key: "0837f275a7d4872793052d29713ae0fa"
server_key_iv: "8a0e886981f16d87ed3ae19c7c5f1406"
server_key_version: "encryptionkey@0"
I20260929 03:09:29.781916 25419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:29.812896 25419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:29.816690 25419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:29.816905 25419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:29.817629 25419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:29.818593 25419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:29.818643 25419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:29.818704 25419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:29.818738 25419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:29.828922 25419 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:42797
I20260929 03:09:29.828945 25547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:42797 every 8 connection(s)
I20260929 03:09:29.830103 25419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:09:29.832720 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 25419
I20260929 03:09:29.832842 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:09:29.833160 8005 external_mini_cluster.cc:1428] Setting key 221dd85f8dfead0db92f07035b10cad0
Sep 29 03:09:29 dist-test-slave-bdps krb5kdc[25044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:09:29.844094 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.832209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:34155 (local address 127.7.209.126:33763)
0929 03:09:29.832363 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:29.832366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:29.833511 (+ 1145us) server_negotiation.cc:407] Connection header received
0929 03:09:29.834752 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:29.834755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:29.834813 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:29.834897 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:29.836336 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.836920 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.838132 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.838355 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.838434 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.840900 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:29.840904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:29.840906 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:29.840931 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:29.842354 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:29.842826 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:29.842829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:29.842830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:29.842904 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:29.843156 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:29.843158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:29.843159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:29.843335 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:29.843406 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:29.843853 (+ 447us) server_negotiation.cc:299] Negotiation successful
0929 03:09:29.843956 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:09:29.845067 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.832700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:33763 (local address 127.7.209.67:34155)
0929 03:09:29.833297 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:29.833369 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:29.834529 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:29.835089 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:29.835097 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:29.835598 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:29.836201 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:29.836219 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.837029 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.837032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:29.838043 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:29.838052 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.838471 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.838474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:29.838626 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.839221 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:09:29.839237 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:29.840748 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:29.842455 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:29.842460 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:29.842473 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:29.842737 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:29.842966 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:29.842968 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:29.842970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:29.843096 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:29.843402 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:29.843408 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:29.843649 (+ 241us) client_negotiation.cc:769] Sending connection context
0929 03:09:29.843826 (+ 177us) client_negotiation.cc:240] Negotiation successful
0929 03:09:29.844098 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":149,"threads_started":1}
I20260929 03:09:29.846323 25548 heartbeater.cc:347] Connected to a master server at 127.7.209.126:33763
I20260929 03:09:29.846577 25548 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:29.847213 25548 heartbeater.cc:511] Master 127.7.209.126:33763 requested a full tablet report, sending...
I20260929 03:09:29.848264 25089 ts_manager.cc:194] Registered new tserver with Master: 5da5e4a0bc0444a3b9fd23e9610e57af (127.7.209.67:42797)
I20260929 03:09:29.848845 25089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:34155
I20260929 03:09:29.858814 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:09:29.867929 25128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.861027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54932 (local address 127.7.209.126:33763)
0929 03:09:29.861139 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:29.861142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:29.861374 (+ 232us) server_negotiation.cc:407] Connection header received
0929 03:09:29.861479 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:29.861481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:29.861518 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:29.861580 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:29.862271 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.862795 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.863621 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.863840 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.863907 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.865176 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:29.865179 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:29.865181 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:29.865203 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:29.866536 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:29.866889 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:29.866892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:29.866893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:29.866962 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:29.867146 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:29.867148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:29.867149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:29.867371 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:29.867475 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:29.867672 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 03:09:29.867794 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 03:09:29.870975 25089 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:09:29.874617 25089 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:09:29.891191 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.884659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:42797 (local address 127.0.0.1:35506)
0929 03:09:29.885986 (+ 1327us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:29.886000 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:29.886086 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:29.886677 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:29.886680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:29.886700 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:29.886809 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:29.886816 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.888194 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.888197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:29.889437 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:29.889446 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.890609 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.890612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:29.890728 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.890774 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 03:09:29.890886 (+ 112us) client_negotiation.cc:240] Negotiation successful
0929 03:09:29.891016 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1266,"mutex_wait_us":61,"thread_start_us":127,"threads_started":1}
I20260929 03:09:29.891259 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.885283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52722 (local address 127.7.209.66:37303)
0929 03:09:29.886878 (+ 1595us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:29.886892 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:29.886915 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:09:29.886985 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:29.886989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:29.887120 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:29.887256 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:29.887614 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.888557 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.890093 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.890685 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.890786 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.890946 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:29.890998 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 03:09:29.891101 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1472,"thread_start_us":135,"threads_started":1}
I20260929 03:09:29.891193 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.884398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:37303 (local address 127.0.0.1:52722)
0929 03:09:29.885893 (+ 1495us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:29.885906 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:29.885990 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:29.887415 (+ 1425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:29.887417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:29.887432 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:29.887506 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:29.887510 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.888743 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.888746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:29.889968 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:29.889975 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.890797 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.890800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:29.890886 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.890909 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 03:09:29.890967 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 03:09:29.891039 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1435,"mutex_wait_us":53,"thread_start_us":121,"threads_started":1}
I20260929 03:09:29.891775 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.884310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:38971 (local address 127.0.0.1:36148)
0929 03:09:29.885309 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:29.885369 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:29.885516 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:29.886391 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:29.886396 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:29.886416 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:29.886556 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:29.886564 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.888168 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.888171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:29.889939 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:29.889950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.891161 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.891164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:29.891248 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.891270 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 03:09:29.891569 (+ 299us) client_negotiation.cc:240] Negotiation successful
0929 03:09:29.891646 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":261,"thread_start_us":98,"threads_started":1}
I20260929 03:09:29.892162 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.884485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.7.209.65:38971)
0929 03:09:29.885501 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:29.885510 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:29.885539 (+ 29us) server_negotiation.cc:407] Connection header received
0929 03:09:29.885781 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:29.885788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:29.886050 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:29.886216 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:29.886674 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.888044 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.890071 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.890881 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.890970 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.891105 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:29.891937 (+ 832us) server_negotiation.cc:299] Negotiation successful
0929 03:09:29.892046 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":891,"thread_start_us":90,"threads_started":1}
I20260929 03:09:29.892601 25564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:29.885747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35506 (local address 127.7.209.67:42797)
0929 03:09:29.886286 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:29.886299 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:29.886325 (+ 26us) server_negotiation.cc:407] Connection header received
0929 03:09:29.886407 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:29.886410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:29.886545 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:29.886692 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:29.886905 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.888053 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.889552 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:29.890280 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:29.890385 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:29.890547 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:29.891287 (+ 740us) server_negotiation.cc:299] Negotiation successful
0929 03:09:29.892456 (+ 1169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":174,"threads_started":1}
I20260929 03:09:29.895017 25214 tablet_service.cc:1467] Processing CreateTablet for tablet 163927fa00034cda985c5c77ebd4b612 (DEFAULT_TABLE table=test-table [id=712a864487f04abd9d62103fb2faac0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:29.895017 25348 tablet_service.cc:1467] Processing CreateTablet for tablet 163927fa00034cda985c5c77ebd4b612 (DEFAULT_TABLE table=test-table [id=712a864487f04abd9d62103fb2faac0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:29.895251 25482 tablet_service.cc:1467] Processing CreateTablet for tablet 163927fa00034cda985c5c77ebd4b612 (DEFAULT_TABLE table=test-table [id=712a864487f04abd9d62103fb2faac0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:29.896265 25348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 163927fa00034cda985c5c77ebd4b612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:29.896265 25482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 163927fa00034cda985c5c77ebd4b612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:29.896271 25214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 163927fa00034cda985c5c77ebd4b612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:29.903034 25565 tablet_bootstrap.cc:492] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a: Bootstrap starting.
I20260929 03:09:29.904675 25567 tablet_bootstrap.cc:492] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af: Bootstrap starting.
I20260929 03:09:29.904675 25566 tablet_bootstrap.cc:492] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd: Bootstrap starting.
I20260929 03:09:29.905547 25565 tablet_bootstrap.cc:654] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:29.906582 25565 log.cc:824] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:29.907157 25567 tablet_bootstrap.cc:654] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:29.907625 25566 tablet_bootstrap.cc:654] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:29.908519 25567 log.cc:824] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:29.908519 25566 log.cc:824] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:29.908862 25565 tablet_bootstrap.cc:492] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a: No bootstrap required, opened a new log
I20260929 03:09:29.909075 25565 ts_tablet_manager.cc:1397] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:09:29.910719 25567 tablet_bootstrap.cc:492] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af: No bootstrap required, opened a new log
I20260929 03:09:29.910751 25566 tablet_bootstrap.cc:492] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd: No bootstrap required, opened a new log
I20260929 03:09:29.910881 25567 ts_tablet_manager.cc:1397] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:09:29.910929 25566 ts_tablet_manager.cc:1397] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260929 03:09:29.914230 25567 raft_consensus.cc:348] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1725ac07d4fec8b55b026383dc36a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38971 } } peers { permanent_uuid: "aff882ff2aa147c5bacf957f6a25cefd" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37303 } } peers { permanent_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42797 } }
I20260929 03:09:29.914155 25565 raft_consensus.cc:348] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1725ac07d4fec8b55b026383dc36a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38971 } } peers { permanent_uuid: "aff882ff2aa147c5bacf957f6a25cefd" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37303 } } peers { permanent_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42797 } }
I20260929 03:09:29.914472 25567 raft_consensus.cc:374] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:29.914474 25565 raft_consensus.cc:374] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:29.914528 25565 raft_consensus.cc:729] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99a1725ac07d4fec8b55b026383dc36a, State: Initialized, Role: FOLLOWER
I20260929 03:09:29.914528 25567 raft_consensus.cc:729] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5da5e4a0bc0444a3b9fd23e9610e57af, State: Initialized, Role: FOLLOWER
I20260929 03:09:29.914898 25566 raft_consensus.cc:348] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1725ac07d4fec8b55b026383dc36a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38971 } } peers { permanent_uuid: "aff882ff2aa147c5bacf957f6a25cefd" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37303 } } peers { permanent_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42797 } }
I20260929 03:09:29.915021 25567 consensus_queue.cc:260] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1725ac07d4fec8b55b026383dc36a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38971 } } peers { permanent_uuid: "aff882ff2aa147c5bacf957f6a25cefd" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37303 } } peers { permanent_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42797 } }
I20260929 03:09:29.915055 25565 consensus_queue.cc:260] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1725ac07d4fec8b55b026383dc36a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38971 } } peers { permanent_uuid: "aff882ff2aa147c5bacf957f6a25cefd" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37303 } } peers { permanent_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42797 } }
I20260929 03:09:29.915148 25566 raft_consensus.cc:374] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:29.915199 25566 raft_consensus.cc:729] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aff882ff2aa147c5bacf957f6a25cefd, State: Initialized, Role: FOLLOWER
I20260929 03:09:29.915606 25566 consensus_queue.cc:260] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1725ac07d4fec8b55b026383dc36a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38971 } } peers { permanent_uuid: "aff882ff2aa147c5bacf957f6a25cefd" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37303 } } peers { permanent_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42797 } }
I20260929 03:09:29.916091 25548 heartbeater.cc:503] Master 127.7.209.126:33763 was elected leader, sending a full tablet report...
I20260929 03:09:29.916101 25280 heartbeater.cc:503] Master 127.7.209.126:33763 was elected leader, sending a full tablet report...
I20260929 03:09:29.916229 25567 ts_tablet_manager.cc:1428] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 03:09:29.916474 25566 ts_tablet_manager.cc:1428] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260929 03:09:29.916813 25414 heartbeater.cc:503] Master 127.7.209.126:33763 was elected leader, sending a full tablet report...
I20260929 03:09:29.917326 25565 ts_tablet_manager.cc:1428] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260929 03:09:30.030841 25571 raft_consensus.cc:482] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:09:30.031126 25571 raft_consensus.cc:504] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1725ac07d4fec8b55b026383dc36a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38971 } } peers { permanent_uuid: "aff882ff2aa147c5bacf957f6a25cefd" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37303 } } peers { permanent_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42797 } }
I20260929 03:09:30.032686 25571 leader_election.cc:288] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99a1725ac07d4fec8b55b026383dc36a (127.7.209.65:38971), aff882ff2aa147c5bacf957f6a25cefd (127.7.209.66:37303)
I20260929 03:09:30.037000 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:30.033254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:51507 (local address 127.7.209.66:37303)
0929 03:09:30.033406 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:30.033411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:30.033427 (+ 16us) server_negotiation.cc:407] Connection header received
0929 03:09:30.033491 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:30.033494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:30.033549 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:30.033788 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:30.034164 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.034895 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.036013 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.036437 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.036550 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:30.036616 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:30.036782 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 03:09:30.036861 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 03:09:30.037000 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:30.032998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:37303 (local address 127.7.209.67:51507)
0929 03:09:30.033139 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:30.033157 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:30.033272 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:30.033829 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:30.033832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:30.033858 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:30.034007 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:30.034013 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.035001 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.035004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:30.035880 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:30.035889 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.036550 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.036554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:30.036661 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:30.036683 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 03:09:30.036748 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 03:09:30.036840 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":2}
I20260929 03:09:30.037186 25574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:30.032998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:38971 (local address 127.7.209.67:53637)
0929 03:09:30.033755 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:30.033768 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:30.033913 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:30.034297 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:30.034300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:30.034316 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:30.034430 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:30.034435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.035248 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.035252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:30.036072 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:30.036081 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.036745 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.036750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:30.036860 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:30.036877 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 03:09:30.036942 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 03:09:30.037008 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":157,"thread_start_us":206,"threads_started":1}
I20260929 03:09:30.037181 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:30.033504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:53637 (local address 127.7.209.65:38971)
0929 03:09:30.033678 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:30.033682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:30.033855 (+ 173us) server_negotiation.cc:407] Connection header received
0929 03:09:30.034031 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:30.034035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:30.034087 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:30.034140 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:30.034549 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.035121 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.036197 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.036635 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.036749 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:30.036804 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:30.036970 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 03:09:30.037059 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 03:09:30.037940 25234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "163927fa00034cda985c5c77ebd4b612" candidate_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a1725ac07d4fec8b55b026383dc36a" is_pre_election: true
I20260929 03:09:30.037943 25368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "163927fa00034cda985c5c77ebd4b612" candidate_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aff882ff2aa147c5bacf957f6a25cefd" is_pre_election: true
I20260929 03:09:30.038358 25234 raft_consensus.cc:2454] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5da5e4a0bc0444a3b9fd23e9610e57af in term 0.
I20260929 03:09:30.038358 25368 raft_consensus.cc:2454] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5da5e4a0bc0444a3b9fd23e9610e57af in term 0.
I20260929 03:09:30.038955 25436 leader_election.cc:302] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5da5e4a0bc0444a3b9fd23e9610e57af, 99a1725ac07d4fec8b55b026383dc36a; no voters:
I20260929 03:09:30.039269 25571 raft_consensus.cc:2781] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:09:30.039354 25571 raft_consensus.cc:482] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:09:30.039403 25571 raft_consensus.cc:3037] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:30.040650 25571 raft_consensus.cc:504] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1725ac07d4fec8b55b026383dc36a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38971 } } peers { permanent_uuid: "aff882ff2aa147c5bacf957f6a25cefd" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37303 } } peers { permanent_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42797 } }
I20260929 03:09:30.041011 25571 leader_election.cc:288] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [CANDIDATE]: Term 1 election: Requested vote from peers 99a1725ac07d4fec8b55b026383dc36a (127.7.209.65:38971), aff882ff2aa147c5bacf957f6a25cefd (127.7.209.66:37303)
I20260929 03:09:30.041404 25368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "163927fa00034cda985c5c77ebd4b612" candidate_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aff882ff2aa147c5bacf957f6a25cefd"
I20260929 03:09:30.041420 25234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "163927fa00034cda985c5c77ebd4b612" candidate_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99a1725ac07d4fec8b55b026383dc36a"
I20260929 03:09:30.041544 25368 raft_consensus.cc:3037] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:30.041572 25234 raft_consensus.cc:3037] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:30.042786 25234 raft_consensus.cc:2454] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5da5e4a0bc0444a3b9fd23e9610e57af in term 1.
I20260929 03:09:30.042891 25368 raft_consensus.cc:2454] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5da5e4a0bc0444a3b9fd23e9610e57af in term 1.
I20260929 03:09:30.043144 25436 leader_election.cc:302] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5da5e4a0bc0444a3b9fd23e9610e57af, 99a1725ac07d4fec8b55b026383dc36a; no voters:
I20260929 03:09:30.043316 25571 raft_consensus.cc:2781] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:30.043568 25571 raft_consensus.cc:686] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 1 LEADER]: Becoming Leader. State: Replica: 5da5e4a0bc0444a3b9fd23e9610e57af, State: Running, Role: LEADER
I20260929 03:09:30.043987 25571 consensus_queue.cc:237] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1725ac07d4fec8b55b026383dc36a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38971 } } peers { permanent_uuid: "aff882ff2aa147c5bacf957f6a25cefd" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37303 } } peers { permanent_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42797 } }
I20260929 03:09:30.047711 25088 catalog_manager.cc:5495] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af reported cstate change: term changed from 0 to 1, leader changed from <none> to 5da5e4a0bc0444a3b9fd23e9610e57af (127.7.209.67). New cstate: current_term: 1 leader_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99a1725ac07d4fec8b55b026383dc36a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aff882ff2aa147c5bacf957f6a25cefd" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37303 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5da5e4a0bc0444a3b9fd23e9610e57af" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42797 } health_report { overall_health: HEALTHY } } }
I20260929 03:09:30.080773 25564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:30.077267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35512 (local address 127.7.209.67:42797)
0929 03:09:30.077415 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:30.077419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:30.077434 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:09:30.077531 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:30.077533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:30.077597 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:30.077734 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:09:30.077959 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.078492 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.079307 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.079519 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.079592 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:30.079843 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:09:30.080385 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:09:30.080483 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:30.080511 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 03:09:30.080566 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260929 03:09:30.081139 25549 tablet.cc:2369] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:09:30.086135 25415 tablet.cc:2369] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:09:30.088079 25368 raft_consensus.cc:1261] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd [term 1 FOLLOWER]: Refusing update from remote peer 5da5e4a0bc0444a3b9fd23e9610e57af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:09:30.088081 25234 raft_consensus.cc:1261] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a [term 1 FOLLOWER]: Refusing update from remote peer 5da5e4a0bc0444a3b9fd23e9610e57af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:09:30.088819 25575 consensus_queue.cc:1035] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [LEADER]: Connected to new peer: Peer: permanent_uuid: "99a1725ac07d4fec8b55b026383dc36a" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 38971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:09:30.089047 25571 consensus_queue.cc:1035] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [LEADER]: Connected to new peer: Peer: permanent_uuid: "aff882ff2aa147c5bacf957f6a25cefd" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 37303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:09:30.097580 25580 mvcc.cc:205] Tried to move back new op lower bound from 7334508011872239616 to 7334508011703271424. Current Snapshot: MvccSnapshot[applied={T|T < 7334508011872239616}]
I20260929 03:09:30.097952 25579 mvcc.cc:205] Tried to move back new op lower bound from 7334508011872239616 to 7334508011703271424. Current Snapshot: MvccSnapshot[applied={T|T < 7334508011872239616}]
I20260929 03:09:30.107499 25088 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:09:30.107774 25088 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:09:30.110816 25088 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 471370f51fab4b78b41728ff3b965290: Sending DeleteTablet for 3 replicas of tablet 163927fa00034cda985c5c77ebd4b612
I20260929 03:09:30.111801 25348 tablet_service.cc:1514] Processing DeleteTablet for tablet 163927fa00034cda985c5c77ebd4b612 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:30 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:52722
I20260929 03:09:30.111812 25482 tablet_service.cc:1514] Processing DeleteTablet for tablet 163927fa00034cda985c5c77ebd4b612 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:30 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:35506
I20260929 03:09:30.112412 25588 tablet_replica.cc:330] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af: stopping tablet replica
I20260929 03:09:30.112481 25214 tablet_service.cc:1514] Processing DeleteTablet for tablet 163927fa00034cda985c5c77ebd4b612 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:30 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:36148
I20260929 03:09:30.112795 25588 raft_consensus.cc:2229] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:09:30.113066 25589 tablet_replica.cc:330] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a: stopping tablet replica
I20260929 03:09:30.113327 25588 raft_consensus.cc:2258] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:30.113571 25589 raft_consensus.cc:2229] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:09:30.114034 25589 raft_consensus.cc:2258] T 163927fa00034cda985c5c77ebd4b612 P 99a1725ac07d4fec8b55b026383dc36a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:30.114254 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 25151
I20260929 03:09:30.116046 25588 ts_tablet_manager.cc:1905] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:09:30.119542 25588 ts_tablet_manager.cc:1918] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:09:30.119645 25588 log.cc:1196] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/163927fa00034cda985c5c77ebd4b612
I20260929 03:09:30.120023 25588 ts_tablet_manager.cc:1939] T 163927fa00034cda985c5c77ebd4b612 P 5da5e4a0bc0444a3b9fd23e9610e57af: Deleting consensus metadata
I20260929 03:09:30.113785 25587 tablet_replica.cc:330] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd: stopping tablet replica
I20260929 03:09:30.121431 25076 catalog_manager.cc:4841] TS 5da5e4a0bc0444a3b9fd23e9610e57af (127.7.209.67:42797): tablet 163927fa00034cda985c5c77ebd4b612 (table test-table [id=712a864487f04abd9d62103fb2faac0c]) successfully deleted
I20260929 03:09:30.121587 25587 raft_consensus.cc:2229] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:09:30.122008 25587 raft_consensus.cc:2258] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:30.124395 25587 ts_tablet_manager.cc:1905] T 163927fa00034cda985c5c77ebd4b612 P aff882ff2aa147c5bacf957f6a25cefd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 03:09:30.124517 25074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:38971 (error 108)
I20260929 03:09:30.125705 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:30.125369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:38971 (local address 127.0.0.1:36152)
0929 03:09:30.125501 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:30.125575 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:38971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 03:09:30.125833 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 25285
W20260929 03:09:30.127805 25074 catalog_manager.cc:4584] TS 99a1725ac07d4fec8b55b026383dc36a (127.7.209.65:38971): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 163927fa00034cda985c5c77ebd4b612: Network error: Client connection negotiation failed: client connection to 127.7.209.65:38971: connect: Connection refused (error 111)
I20260929 03:09:30.132041 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 25419
I20260929 03:09:30.133853 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:30.132175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:37303 (local address 127.0.0.1:52726)
0929 03:09:30.133693 (+ 1518us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:30.133775 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.66:37303: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1415}
I20260929 03:09:30.137279 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 25057
2026-09-29T03:09:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2366 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T03:09:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:30Z Disabled control of system clock
I20260929 03:09:30.169641 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:39323
--webserver_interface=127.7.209.126
--webserver_port=0
--builtin_ntp_servers=127.7.209.84:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:39323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 03:09:30.290596 25594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:30.290871 25594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:30.290921 25594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:30.294853 25594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:30.294919 25594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:30.294941 25594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:30.294962 25594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:30.294981 25594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:30.299690 25594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:39323
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:39323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:30.300904 25594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:30.302492 25594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:30.311621 25599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:30.311651 25600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:30.311805 25602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:30.311975 25594 server_base.cc:1034] running on GCE node
I20260929 03:09:30.312588 25594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:30.313856 25594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:30.315011 25594 hybrid_clock.cc:648] HybridClock initialized: now 1790651370314992 us; error 33 us; skew 500 ppm
I20260929 03:09:30.316962 25594 webserver.cc:460] Webserver started at http://127.7.209.126:35275/ using document root <none> and password file <none>
I20260929 03:09:30.317590 25594 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:30.317689 25594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:30.317920 25594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:30.320180 25594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "82efd22177184c099615c05609ef1a74"
format_stamp: "Formatted at 2026-09-29 03:09:30 on dist-test-slave-bdps"
server_key: "238ddfc18cfb049350b065a9a8c56d27"
server_key_iv: "1cbe25c5b89f3998dee6712feea467f1"
server_key_version: "encryptionkey@0"
I20260929 03:09:30.320755 25594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "82efd22177184c099615c05609ef1a74"
format_stamp: "Formatted at 2026-09-29 03:09:30 on dist-test-slave-bdps"
server_key: "238ddfc18cfb049350b065a9a8c56d27"
server_key_iv: "1cbe25c5b89f3998dee6712feea467f1"
server_key_version: "encryptionkey@0"
I20260929 03:09:30.324633 25594 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 03:09:30.327100 25608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:30.328356 25594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 03:09:30.328485 25594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "82efd22177184c099615c05609ef1a74"
format_stamp: "Formatted at 2026-09-29 03:09:30 on dist-test-slave-bdps"
server_key: "238ddfc18cfb049350b065a9a8c56d27"
server_key_iv: "1cbe25c5b89f3998dee6712feea467f1"
server_key_version: "encryptionkey@0"
I20260929 03:09:30.328581 25594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:30.360011 25594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:30.361053 25594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:30.361310 25594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:30.369781 25594 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:39323
I20260929 03:09:30.369799 25660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:39323 every 8 connection(s)
I20260929 03:09:30.371045 25594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:30.374998 25661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:30.375450 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 25594
I20260929 03:09:30.375591 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:30.375991 8005 external_mini_cluster.cc:1428] Setting key 09a7f5eba6d12eb97a9a4f8382ef470d
I20260929 03:09:30.382129 25661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74: Bootstrap starting.
I20260929 03:09:30.384686 25661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:30.385637 25661 log.cc:824] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:30.386444 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:30.377717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34344 (local address 127.7.209.126:39323)
0929 03:09:30.378480 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:30.378497 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:30.378539 (+ 42us) server_negotiation.cc:407] Connection header received
0929 03:09:30.379289 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:30.379309 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:30.379784 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:30.380147 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:30.380629 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.381711 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.382695 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.383206 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.383370 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:30.384265 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:30.384271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:30.384286 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:30.384722 (+ 436us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:30.384788 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:30.384816 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:30.384909 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:30.385083 (+ 174us) server_negotiation.cc:299] Negotiation successful
0929 03:09:30.385394 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":162,"threads_started":1}
I20260929 03:09:30.388010 25661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74: No bootstrap required, opened a new log
I20260929 03:09:30.392127 25661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82efd22177184c099615c05609ef1a74" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39323 } }
I20260929 03:09:30.392396 25661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:30.392457 25661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82efd22177184c099615c05609ef1a74, State: Initialized, Role: FOLLOWER
I20260929 03:09:30.392931 25661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82efd22177184c099615c05609ef1a74" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39323 } }
I20260929 03:09:30.393081 25661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:30.393149 25661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:30.393254 25661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:30.394345 25661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82efd22177184c099615c05609ef1a74" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39323 } }
I20260929 03:09:30.394711 25661 leader_election.cc:302] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82efd22177184c099615c05609ef1a74; no voters:
I20260929 03:09:30.395031 25661 leader_election.cc:288] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:30.395140 25666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:30.395406 25666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [term 1 LEADER]: Becoming Leader. State: Replica: 82efd22177184c099615c05609ef1a74, State: Running, Role: LEADER
I20260929 03:09:30.395714 25666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82efd22177184c099615c05609ef1a74" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39323 } }
I20260929 03:09:30.396363 25661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:30.397421 25668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82efd22177184c099615c05609ef1a74. Latest consensus state: current_term: 1 leader_uuid: "82efd22177184c099615c05609ef1a74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82efd22177184c099615c05609ef1a74" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39323 } } }
I20260929 03:09:30.397558 25668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:30.397810 25667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82efd22177184c099615c05609ef1a74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82efd22177184c099615c05609ef1a74" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39323 } } }
I20260929 03:09:30.397910 25667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:30.398607 25675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:30.401878 25675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:30.408522 25675 catalog_manager.cc:1347] Generated new cluster ID: 6daa5055ac17410f97f205f36a920845
I20260929 03:09:30.408600 25675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:30.441869 25675 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:30.443359 25675 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:30.452580 25675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74: Generated new TSK 0
I20260929 03:09:30.453579 25675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:30.466149 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39323
--builtin_ntp_servers=127.7.209.84:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 03:09:30.590590 25685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:30.590884 25685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:30.590938 25685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:30.595088 25685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:30.595162 25685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:30.595237 25685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:09:30.600584 25685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:30.601883 25685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:30.603497 25685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:30.613732 25690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:30.613739 25693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:30.614061 25685 server_base.cc:1034] running on GCE node
W20260929 03:09:30.614058 25691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:30.614486 25685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:30.615129 25685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:30.616297 25685 hybrid_clock.cc:648] HybridClock initialized: now 1790651370616277 us; error 32 us; skew 500 ppm
I20260929 03:09:30.618237 25685 webserver.cc:460] Webserver started at http://127.7.209.65:44115/ using document root <none> and password file <none>
I20260929 03:09:30.618848 25685 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:30.618916 25685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:30.619133 25685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:30.621318 25685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "1571c7b085484b67b6a842e91a3cc82f"
format_stamp: "Formatted at 2026-09-29 03:09:30 on dist-test-slave-bdps"
server_key: "11e04756de2c44d0dbab719319184b1b"
server_key_iv: "93bb35e7c9df2b5c39f9a4ce37eb46ff"
server_key_version: "encryptionkey@0"
I20260929 03:09:30.621902 25685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "1571c7b085484b67b6a842e91a3cc82f"
format_stamp: "Formatted at 2026-09-29 03:09:30 on dist-test-slave-bdps"
server_key: "11e04756de2c44d0dbab719319184b1b"
server_key_iv: "93bb35e7c9df2b5c39f9a4ce37eb46ff"
server_key_version: "encryptionkey@0"
I20260929 03:09:30.625761 25685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:09:30.628479 25699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:30.629930 25685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:30.630048 25685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "1571c7b085484b67b6a842e91a3cc82f"
format_stamp: "Formatted at 2026-09-29 03:09:30 on dist-test-slave-bdps"
server_key: "11e04756de2c44d0dbab719319184b1b"
server_key_iv: "93bb35e7c9df2b5c39f9a4ce37eb46ff"
server_key_version: "encryptionkey@0"
I20260929 03:09:30.630173 25685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:30.652216 25685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:30.653163 25685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:30.653374 25685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:30.654187 25685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:30.655385 25685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:30.655439 25685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:30.655503 25685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:30.655529 25685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:30.665972 25685 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:35193
I20260929 03:09:30.666064 25812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:35193 every 8 connection(s)
I20260929 03:09:30.667236 25685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:09:30.674806 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 25685
I20260929 03:09:30.674959 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:09:30.675346 8005 external_mini_cluster.cc:1428] Setting key 3bca6d7cf4066efaf1815bb933326131
I20260929 03:09:30.677594 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39323
--builtin_ntp_servers=127.7.209.84:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 03:09:30.678740 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:30.669770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:47985 (local address 127.7.209.126:39323)
0929 03:09:30.669963 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:30.669968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:30.671415 (+ 1447us) server_negotiation.cc:407] Connection header received
0929 03:09:30.673048 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:30.673051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:30.673111 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:30.673201 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:30.674426 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.675324 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.676589 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.676833 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.676932 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:30.678225 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:30.678228 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:30.678230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:30.678243 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:30.678265 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:30.678275 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:30.678338 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:30.678496 (+ 158us) server_negotiation.cc:299] Negotiation successful
0929 03:09:30.678572 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:09:30.679788 25815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:30.670385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39323 (local address 127.7.209.65:47985)
0929 03:09:30.671235 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:30.671287 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:30.672793 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:30.673392 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:30.673400 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:30.673508 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:30.674280 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:30.674301 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.675469 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.675471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:30.676459 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:30.676469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.676993 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.676998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:30.677185 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:30.677944 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:09:30.677946 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:30.677996 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:09:30.678070 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:30.678347 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:30.678357 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:30.678364 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 03:09:30.678477 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 03:09:30.678842 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":143,"threads_started":1}
I20260929 03:09:30.681109 25813 heartbeater.cc:347] Connected to a master server at 127.7.209.126:39323
I20260929 03:09:30.681401 25813 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:30.682103 25813 heartbeater.cc:511] Master 127.7.209.126:39323 requested a full tablet report, sending...
I20260929 03:09:30.683883 25625 ts_manager.cc:194] Registered new tserver with Master: 1571c7b085484b67b6a842e91a3cc82f (127.7.209.65:35193)
I20260929 03:09:30.685192 25625 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.209.65:47985
W20260929 03:09:30.800765 25816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:30.801091 25816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:30.801148 25816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:30.805151 25816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:30.805204 25816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:30.805279 25816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:09:30.810096 25816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:30.811355 25816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:30.812963 25816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:30.823486 25824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:30.823504 25822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:30.823589 25816 server_base.cc:1034] running on GCE node
W20260929 03:09:30.823486 25821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:30.824162 25816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:30.824869 25816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:30.826023 25816 hybrid_clock.cc:648] HybridClock initialized: now 1790651370826002 us; error 34 us; skew 500 ppm
I20260929 03:09:30.828104 25816 webserver.cc:460] Webserver started at http://127.7.209.66:38339/ using document root <none> and password file <none>
I20260929 03:09:30.828814 25816 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:30.828887 25816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:30.829121 25816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:30.831589 25816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9"
format_stamp: "Formatted at 2026-09-29 03:09:30 on dist-test-slave-bdps"
server_key: "169f246506aedae6feae86ec163ce0ca"
server_key_iv: "11f40470a10261327b9eeb947e5847a5"
server_key_version: "encryptionkey@0"
I20260929 03:09:30.832229 25816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9"
format_stamp: "Formatted at 2026-09-29 03:09:30 on dist-test-slave-bdps"
server_key: "169f246506aedae6feae86ec163ce0ca"
server_key_iv: "11f40470a10261327b9eeb947e5847a5"
server_key_version: "encryptionkey@0"
I20260929 03:09:30.836654 25816 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 03:09:30.839407 25830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:30.840734 25816 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 03:09:30.840847 25816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9"
format_stamp: "Formatted at 2026-09-29 03:09:30 on dist-test-slave-bdps"
server_key: "169f246506aedae6feae86ec163ce0ca"
server_key_iv: "11f40470a10261327b9eeb947e5847a5"
server_key_version: "encryptionkey@0"
I20260929 03:09:30.840945 25816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:30.859283 25816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:30.860069 25816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:30.860261 25816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:30.860929 25816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:30.862005 25816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:30.862054 25816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:30.862109 25816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:30.862138 25816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:30.871732 25816 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:39895
I20260929 03:09:30.871767 25943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:39895 every 8 connection(s)
I20260929 03:09:30.872756 25816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:09:30.875487 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 25816
I20260929 03:09:30.875635 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:09:30.876039 8005 external_mini_cluster.cc:1428] Setting key 3cb50e4f2c84f0ccd484acc63c16cae0
I20260929 03:09:30.878207 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39323
--builtin_ntp_servers=127.7.209.84:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 03:09:30.882894 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:30.874855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:55497 (local address 127.7.209.126:39323)
0929 03:09:30.875010 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:30.875014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:30.876321 (+ 1307us) server_negotiation.cc:407] Connection header received
0929 03:09:30.877997 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:30.878000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:30.878060 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:30.878134 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:30.879172 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.879741 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.880929 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.881156 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.881248 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:30.882402 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:30.882405 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:30.882407 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:30.882419 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:30.882439 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:30.882447 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:30.882507 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:30.882649 (+ 142us) server_negotiation.cc:299] Negotiation successful
0929 03:09:30.882723 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:09:30.883803 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:30.875333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39323 (local address 127.7.209.66:55497)
0929 03:09:30.876158 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:30.876206 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:30.877785 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:30.878363 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:30.878374 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:30.878474 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:30.879033 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:30.879048 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.879863 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.879866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:30.880814 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:30.880823 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:30.881303 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:30.881307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:30.881473 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:30.882129 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:09:30.882130 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:30.882177 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:09:30.882254 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:30.882516 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:30.882526 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:30.882532 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 03:09:30.882643 (+ 111us) client_negotiation.cc:240] Negotiation successful
0929 03:09:30.882945 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":162,"threads_started":1}
I20260929 03:09:30.885053 25944 heartbeater.cc:347] Connected to a master server at 127.7.209.126:39323
I20260929 03:09:30.885286 25944 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:30.885922 25944 heartbeater.cc:511] Master 127.7.209.126:39323 requested a full tablet report, sending...
I20260929 03:09:30.887095 25625 ts_manager.cc:194] Registered new tserver with Master: bfc1aeb863a54ffda5f0a9e0dd2104e9 (127.7.209.66:39895)
I20260929 03:09:30.887935 25625 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.209.66:55497
W20260929 03:09:31.006347 25947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:31.006680 25947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:31.006721 25947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:31.010794 25947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:31.010847 25947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:31.010902 25947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:09:31.015861 25947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:39323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:31.017194 25947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:31.018921 25947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:31.029469 25952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:31.029678 25947 server_base.cc:1034] running on GCE node
W20260929 03:09:31.029510 25953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:31.029523 25955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:31.030310 25947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:31.031091 25947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:31.032264 25947 hybrid_clock.cc:648] HybridClock initialized: now 1790651371032242 us; error 37 us; skew 500 ppm
I20260929 03:09:31.034543 25947 webserver.cc:460] Webserver started at http://127.7.209.67:41677/ using document root <none> and password file <none>
I20260929 03:09:31.035249 25947 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:31.035315 25947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:31.035549 25947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:31.038030 25947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "c06414aa02854b978749363a69a9b963"
format_stamp: "Formatted at 2026-09-29 03:09:31 on dist-test-slave-bdps"
server_key: "a0cf8d116326b81cb3ee7d9668646fc9"
server_key_iv: "007ce040c59821d16496f13e1dcf5f82"
server_key_version: "encryptionkey@0"
I20260929 03:09:31.038642 25947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "c06414aa02854b978749363a69a9b963"
format_stamp: "Formatted at 2026-09-29 03:09:31 on dist-test-slave-bdps"
server_key: "a0cf8d116326b81cb3ee7d9668646fc9"
server_key_iv: "007ce040c59821d16496f13e1dcf5f82"
server_key_version: "encryptionkey@0"
I20260929 03:09:31.042801 25947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:09:31.045518 25961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:31.046895 25947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:31.047003 25947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "c06414aa02854b978749363a69a9b963"
format_stamp: "Formatted at 2026-09-29 03:09:31 on dist-test-slave-bdps"
server_key: "a0cf8d116326b81cb3ee7d9668646fc9"
server_key_iv: "007ce040c59821d16496f13e1dcf5f82"
server_key_version: "encryptionkey@0"
I20260929 03:09:31.047099 25947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:31.059060 25947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:31.059772 25947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:31.059942 25947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:31.060554 25947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:31.061587 25947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:31.061666 25947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:31.061723 25947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:31.061754 25947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:31.071647 25947 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:36973
I20260929 03:09:31.071684 26074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:36973 every 8 connection(s)
I20260929 03:09:31.072741 25947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:09:31.076093 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 25947
I20260929 03:09:31.076220 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:09:31.076577 8005 external_mini_cluster.cc:1428] Setting key 8ae5a73b490c923699c457bc424e45e3
I20260929 03:09:31.083353 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.074931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:44807 (local address 127.7.209.126:39323)
0929 03:09:31.075126 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:31.075132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:31.076462 (+ 1330us) server_negotiation.cc:407] Connection header received
0929 03:09:31.078229 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:31.078233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:31.078298 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:31.078378 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:31.079555 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.080141 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.081343 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.081580 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.081699 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.082860 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:31.082863 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:31.082866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:31.082878 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:31.082898 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:31.082906 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:31.082966 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:31.083116 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 03:09:31.083189 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 03:09:31.084307 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.075467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:39323 (local address 127.7.209.67:44807)
0929 03:09:31.076277 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:31.076337 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:31.077993 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:31.078595 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:31.078603 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:31.078711 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:31.079405 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:31.079423 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.080268 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.080271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:31.081215 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:31.081223 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.081751 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.081755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:31.081927 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.082586 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:09:31.082587 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:31.082635 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:09:31.082705 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:31.082975 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:31.082984 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:31.082991 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 03:09:31.083108 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 03:09:31.083427 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":131,"threads_started":1}
I20260929 03:09:31.085409 26075 heartbeater.cc:347] Connected to a master server at 127.7.209.126:39323
I20260929 03:09:31.085654 26075 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:31.086187 26075 heartbeater.cc:511] Master 127.7.209.126:39323 requested a full tablet report, sending...
I20260929 03:09:31.087215 25625 ts_manager.cc:194] Registered new tserver with Master: c06414aa02854b978749363a69a9b963 (127.7.209.67:36973)
I20260929 03:09:31.087836 25625 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.209.67:44807
I20260929 03:09:31.090488 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:09:31.096510 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.092707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34360 (local address 127.7.209.126:39323)
0929 03:09:31.092837 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:31.092842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:31.093027 (+ 185us) server_negotiation.cc:407] Connection header received
0929 03:09:31.093165 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:31.093169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:31.093215 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:31.093282 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:31.093565 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.094137 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.094996 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.095215 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.095303 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.096073 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:31.096077 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:31.096080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:31.096093 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:31.096114 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:31.096122 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:31.096180 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:31.096288 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 03:09:31.096346 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 03:09:31.099298 25625 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:09:31.102756 25625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:09:31.118736 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.112449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43738 (local address 127.7.209.65:35193)
0929 03:09:31.113544 (+ 1095us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:31.113559 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:31.113590 (+ 31us) server_negotiation.cc:407] Connection header received
0929 03:09:31.113713 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:31.113720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:31.113971 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:31.114150 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:31.114462 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.115816 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.117128 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.117953 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.118067 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.118336 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:31.118397 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 03:09:31.118527 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":956,"thread_start_us":172,"threads_started":1}
I20260929 03:09:31.119239 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.112588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.7.209.66:39895)
0929 03:09:31.113695 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:31.113707 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:31.113730 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:09:31.113806 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:31.113810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:31.113968 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:31.114150 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:31.114643 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.115977 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.117482 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.118244 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.118347 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.118552 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:31.118960 (+ 408us) server_negotiation.cc:299] Negotiation successful
0929 03:09:31.119068 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":970,"thread_start_us":161,"threads_started":1}
I20260929 03:09:31.119414 26090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.113453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:36973 (local address 127.0.0.1:41606)
0929 03:09:31.113780 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:31.113792 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:31.113883 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:31.114886 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:31.114888 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:31.114902 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:31.114977 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:31.114981 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.116289 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.116292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:31.117409 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:31.117417 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.119026 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.119029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:31.119116 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.119138 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 03:09:31.119196 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 03:09:31.119279 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":3,"thread_start_us":90,"threads_started":1}
I20260929 03:09:31.119470 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.112231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:35193 (local address 127.0.0.1:43738)
0929 03:09:31.113147 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:31.113189 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:31.113337 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:31.114142 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:31.114149 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:31.114175 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:31.114335 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:31.114342 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.115931 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.115934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:31.117000 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:31.117009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.118070 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.118074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:31.118182 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.118229 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 03:09:31.118347 (+ 118us) client_negotiation.cc:240] Negotiation successful
0929 03:09:31.118487 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":1,"spinlock_wait_cycles":6656,"thread_start_us":99,"threads_started":1}
I20260929 03:09:31.119874 26086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.112300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:39895 (local address 127.0.0.1:35060)
0929 03:09:31.113145 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:31.113186 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:31.113337 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:31.114353 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:31.114356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:31.114374 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:31.114484 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:31.114516 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.116074 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.116077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:31.117337 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:31.117347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.118614 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.118617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:31.118698 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.118721 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 03:09:31.118801 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 03:09:31.118874 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"mutex_wait_us":26,"spinlock_wait_cycles":23040,"thread_start_us":118,"threads_started":1}
I20260929 03:09:31.121104 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.113756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41606 (local address 127.7.209.67:36973)
0929 03:09:31.114229 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:31.114251 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:31.114276 (+ 25us) server_negotiation.cc:407] Connection header received
0929 03:09:31.114358 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:31.114363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:31.114578 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:31.114734 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:31.115113 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.116177 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.117526 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.118731 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.119030 (+ 299us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.119246 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:31.119293 (+ 47us) server_negotiation.cc:299] Negotiation successful
0929 03:09:31.119393 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":169,"threads_started":1}
I20260929 03:09:31.124380 26009 tablet_service.cc:1467] Processing CreateTablet for tablet a4936bd5a1974781b779ddce4d62eb80 (DEFAULT_TABLE table=test-table [id=ac8afc91dab24ee0bc327b2a0e1e32a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:31.124397 25878 tablet_service.cc:1467] Processing CreateTablet for tablet a4936bd5a1974781b779ddce4d62eb80 (DEFAULT_TABLE table=test-table [id=ac8afc91dab24ee0bc327b2a0e1e32a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:31.125530 26009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4936bd5a1974781b779ddce4d62eb80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:31.125532 25878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4936bd5a1974781b779ddce4d62eb80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:31.126001 25747 tablet_service.cc:1467] Processing CreateTablet for tablet a4936bd5a1974781b779ddce4d62eb80 (DEFAULT_TABLE table=test-table [id=ac8afc91dab24ee0bc327b2a0e1e32a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:31.126956 25747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4936bd5a1974781b779ddce4d62eb80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:31.132365 26092 tablet_bootstrap.cc:492] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963: Bootstrap starting.
I20260929 03:09:31.133904 26094 tablet_bootstrap.cc:492] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f: Bootstrap starting.
I20260929 03:09:31.133929 26093 tablet_bootstrap.cc:492] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9: Bootstrap starting.
I20260929 03:09:31.135380 26092 tablet_bootstrap.cc:654] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:31.136075 26094 tablet_bootstrap.cc:654] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:31.136088 26093 tablet_bootstrap.cc:654] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:31.136358 26092 log.cc:824] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:31.136825 26094 log.cc:824] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:31.136857 26093 log.cc:824] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:31.138484 26092 tablet_bootstrap.cc:492] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963: No bootstrap required, opened a new log
I20260929 03:09:31.138645 26092 ts_tablet_manager.cc:1397] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 03:09:31.138768 26093 tablet_bootstrap.cc:492] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9: No bootstrap required, opened a new log
I20260929 03:09:31.138994 26093 ts_tablet_manager.cc:1397] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260929 03:09:31.139065 26094 tablet_bootstrap.cc:492] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f: No bootstrap required, opened a new log
I20260929 03:09:31.139222 26094 ts_tablet_manager.cc:1397] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:09:31.141861 26092 raft_consensus.cc:348] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1571c7b085484b67b6a842e91a3cc82f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35193 } } peers { permanent_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39895 } } peers { permanent_uuid: "c06414aa02854b978749363a69a9b963" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 36973 } }
I20260929 03:09:31.142079 26092 raft_consensus.cc:374] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:31.142115 26092 raft_consensus.cc:729] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c06414aa02854b978749363a69a9b963, State: Initialized, Role: FOLLOWER
I20260929 03:09:31.142503 26092 consensus_queue.cc:260] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1571c7b085484b67b6a842e91a3cc82f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35193 } } peers { permanent_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39895 } } peers { permanent_uuid: "c06414aa02854b978749363a69a9b963" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 36973 } }
I20260929 03:09:31.143106 26075 heartbeater.cc:503] Master 127.7.209.126:39323 was elected leader, sending a full tablet report...
I20260929 03:09:31.143348 26094 raft_consensus.cc:348] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1571c7b085484b67b6a842e91a3cc82f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35193 } } peers { permanent_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39895 } } peers { permanent_uuid: "c06414aa02854b978749363a69a9b963" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 36973 } }
I20260929 03:09:31.143595 26092 ts_tablet_manager.cc:1428] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 03:09:31.143594 26094 raft_consensus.cc:374] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:31.143656 26094 raft_consensus.cc:729] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1571c7b085484b67b6a842e91a3cc82f, State: Initialized, Role: FOLLOWER
I20260929 03:09:31.144155 26094 consensus_queue.cc:260] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1571c7b085484b67b6a842e91a3cc82f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35193 } } peers { permanent_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39895 } } peers { permanent_uuid: "c06414aa02854b978749363a69a9b963" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 36973 } }
I20260929 03:09:31.144657 26093 raft_consensus.cc:348] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1571c7b085484b67b6a842e91a3cc82f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35193 } } peers { permanent_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39895 } } peers { permanent_uuid: "c06414aa02854b978749363a69a9b963" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 36973 } }
I20260929 03:09:31.144868 25813 heartbeater.cc:503] Master 127.7.209.126:39323 was elected leader, sending a full tablet report...
I20260929 03:09:31.144908 26093 raft_consensus.cc:374] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:31.144953 26093 raft_consensus.cc:729] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc1aeb863a54ffda5f0a9e0dd2104e9, State: Initialized, Role: FOLLOWER
I20260929 03:09:31.145160 26094 ts_tablet_manager.cc:1428] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 03:09:31.145351 26093 consensus_queue.cc:260] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1571c7b085484b67b6a842e91a3cc82f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35193 } } peers { permanent_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39895 } } peers { permanent_uuid: "c06414aa02854b978749363a69a9b963" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 36973 } }
I20260929 03:09:31.145951 25944 heartbeater.cc:503] Master 127.7.209.126:39323 was elected leader, sending a full tablet report...
I20260929 03:09:31.146281 26093 ts_tablet_manager.cc:1428] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
W20260929 03:09:31.168655 25814 tablet.cc:2369] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:09:31.204034 26098 raft_consensus.cc:482] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:09:31.204305 26098 raft_consensus.cc:504] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1571c7b085484b67b6a842e91a3cc82f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35193 } } peers { permanent_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39895 } } peers { permanent_uuid: "c06414aa02854b978749363a69a9b963" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 36973 } }
I20260929 03:09:31.205837 26098 leader_election.cc:288] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1571c7b085484b67b6a842e91a3cc82f (127.7.209.65:35193), bfc1aeb863a54ffda5f0a9e0dd2104e9 (127.7.209.66:39895)
I20260929 03:09:31.209599 26088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.206373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:48887 (local address 127.7.209.65:35193)
0929 03:09:31.206510 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:31.206514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:31.206525 (+ 11us) server_negotiation.cc:407] Connection header received
0929 03:09:31.206582 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:31.206584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:31.206630 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:31.206705 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:31.207016 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.207563 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.208638 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.209061 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.209173 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.209234 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:31.209397 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 03:09:31.209473 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:09:31.209599 26077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.206080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:35193 (local address 127.7.209.67:48887)
0929 03:09:31.206246 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:31.206264 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:31.206375 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:31.206740 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:31.206744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:31.206766 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:31.206893 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:31.206899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.207685 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.207688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:31.208505 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:31.208514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.209173 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.209177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:31.209282 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.209303 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 03:09:31.209371 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 03:09:31.209449 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260929 03:09:31.209851 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.206243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:42893 (local address 127.7.209.66:39895)
0929 03:09:31.206401 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:31.206407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:31.206843 (+ 436us) server_negotiation.cc:407] Connection header received
0929 03:09:31.206953 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:31.206956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:31.207008 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:31.207074 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:31.207357 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.207870 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.208871 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.209256 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.209379 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.209428 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:31.209597 (+ 169us) server_negotiation.cc:299] Negotiation successful
0929 03:09:31.209704 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 03:09:31.210206 26101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.206087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:39895 (local address 127.7.209.67:42893)
0929 03:09:31.206739 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:31.206751 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:31.206857 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:31.207092 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:31.207095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:31.207110 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:31.207241 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:31.207246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.207978 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.207982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:31.208748 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:31.208757 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.209390 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.209393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:31.209487 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.209506 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 03:09:31.209570 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 03:09:31.209662 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":219,"thread_start_us":180,"threads_started":1}
I20260929 03:09:31.210330 25767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4936bd5a1974781b779ddce4d62eb80" candidate_uuid: "c06414aa02854b978749363a69a9b963" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1571c7b085484b67b6a842e91a3cc82f" is_pre_election: true
I20260929 03:09:31.210739 25767 raft_consensus.cc:2454] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06414aa02854b978749363a69a9b963 in term 0.
I20260929 03:09:31.210897 25898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4936bd5a1974781b779ddce4d62eb80" candidate_uuid: "c06414aa02854b978749363a69a9b963" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9" is_pre_election: true
I20260929 03:09:31.211261 25898 raft_consensus.cc:2454] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c06414aa02854b978749363a69a9b963 in term 0.
I20260929 03:09:31.211352 25964 leader_election.cc:302] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1571c7b085484b67b6a842e91a3cc82f, c06414aa02854b978749363a69a9b963; no voters:
I20260929 03:09:31.211669 26098 raft_consensus.cc:2781] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:09:31.211759 26098 raft_consensus.cc:482] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:09:31.211807 26098 raft_consensus.cc:3037] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:31.213167 26098 raft_consensus.cc:504] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1571c7b085484b67b6a842e91a3cc82f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35193 } } peers { permanent_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39895 } } peers { permanent_uuid: "c06414aa02854b978749363a69a9b963" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 36973 } }
I20260929 03:09:31.213558 26098 leader_election.cc:288] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [CANDIDATE]: Term 1 election: Requested vote from peers 1571c7b085484b67b6a842e91a3cc82f (127.7.209.65:35193), bfc1aeb863a54ffda5f0a9e0dd2104e9 (127.7.209.66:39895)
I20260929 03:09:31.213941 25767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4936bd5a1974781b779ddce4d62eb80" candidate_uuid: "c06414aa02854b978749363a69a9b963" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1571c7b085484b67b6a842e91a3cc82f"
I20260929 03:09:31.213951 25898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4936bd5a1974781b779ddce4d62eb80" candidate_uuid: "c06414aa02854b978749363a69a9b963" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9"
I20260929 03:09:31.214075 25767 raft_consensus.cc:3037] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:31.214068 25898 raft_consensus.cc:3037] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:31.215221 25767 raft_consensus.cc:2454] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06414aa02854b978749363a69a9b963 in term 1.
I20260929 03:09:31.215231 25898 raft_consensus.cc:2454] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c06414aa02854b978749363a69a9b963 in term 1.
I20260929 03:09:31.215505 25964 leader_election.cc:302] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1571c7b085484b67b6a842e91a3cc82f, c06414aa02854b978749363a69a9b963; no voters:
I20260929 03:09:31.215682 26098 raft_consensus.cc:2781] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:31.215953 26098 raft_consensus.cc:686] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 1 LEADER]: Becoming Leader. State: Replica: c06414aa02854b978749363a69a9b963, State: Running, Role: LEADER
I20260929 03:09:31.216320 26098 consensus_queue.cc:237] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1571c7b085484b67b6a842e91a3cc82f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35193 } } peers { permanent_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39895 } } peers { permanent_uuid: "c06414aa02854b978749363a69a9b963" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 36973 } }
I20260929 03:09:31.219954 25625 catalog_manager.cc:5495] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 reported cstate change: term changed from 0 to 1, leader changed from <none> to c06414aa02854b978749363a69a9b963 (127.7.209.67). New cstate: current_term: 1 leader_uuid: "c06414aa02854b978749363a69a9b963" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1571c7b085484b67b6a842e91a3cc82f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06414aa02854b978749363a69a9b963" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 36973 } health_report { overall_health: HEALTHY } } }
I20260929 03:09:31.246904 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.243301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41620 (local address 127.7.209.67:36973)
0929 03:09:31.243432 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:31.243436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:31.243450 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:09:31.243546 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:31.243549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:31.243612 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:31.243713 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:09:31.243927 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.244514 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.245425 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:31.245676 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:31.245787 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:31.246027 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:09:31.246571 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:09:31.246667 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:31.246695 (+ 28us) server_negotiation.cc:299] Negotiation successful
0929 03:09:31.246751 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:09:31.254006 25898 raft_consensus.cc:1261] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9 [term 1 FOLLOWER]: Refusing update from remote peer c06414aa02854b978749363a69a9b963: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:09:31.254006 25767 raft_consensus.cc:1261] T a4936bd5a1974781b779ddce4d62eb80 P 1571c7b085484b67b6a842e91a3cc82f [term 1 FOLLOWER]: Refusing update from remote peer c06414aa02854b978749363a69a9b963: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:09:31.254763 26102 consensus_queue.cc:1035] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1571c7b085484b67b6a842e91a3cc82f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 35193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:09:31.254963 26098 consensus_queue.cc:1035] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfc1aeb863a54ffda5f0a9e0dd2104e9" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:09:31.262460 26105 mvcc.cc:205] Tried to move back new op lower bound from 7334508016648163328 to 7334508016505044992. Current Snapshot: MvccSnapshot[applied={T|T < 7334508016648163328}]
I20260929 03:09:31.266191 26107 mvcc.cc:205] Tried to move back new op lower bound from 7334508016648163328 to 7334508016505044992. Current Snapshot: MvccSnapshot[applied={T|T < 7334508016648163328}]
I20260929 03:09:31.272194 25623 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:09:31.272408 25623 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:09:31.275537 25623 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82efd22177184c099615c05609ef1a74: Sending DeleteTablet for 3 replicas of tablet a4936bd5a1974781b779ddce4d62eb80
I20260929 03:09:31.276747 25878 tablet_service.cc:1514] Processing DeleteTablet for tablet a4936bd5a1974781b779ddce4d62eb80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:31 UTC) from {username='slave'} at 127.0.0.1:35060
I20260929 03:09:31.276966 26009 tablet_service.cc:1514] Processing DeleteTablet for tablet a4936bd5a1974781b779ddce4d62eb80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:31 UTC) from {username='slave'} at 127.0.0.1:41606
I20260929 03:09:31.277657 26115 tablet_replica.cc:330] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963: stopping tablet replica
I20260929 03:09:31.278013 26115 raft_consensus.cc:2229] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:09:31.278395 26115 raft_consensus.cc:2258] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:31.278713 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 25685
I20260929 03:09:31.278791 26114 tablet_replica.cc:330] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9: stopping tablet replica
I20260929 03:09:31.281544 26114 raft_consensus.cc:2229] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:09:31.281935 26114 raft_consensus.cc:2258] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:31.283761 26114 ts_tablet_manager.cc:1905] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:09:31.286899 26114 ts_tablet_manager.cc:1918] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:09:31.286995 26114 log.cc:1196] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/a4936bd5a1974781b779ddce4d62eb80
W20260929 03:09:31.287111 25611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:35193 (error 108)
I20260929 03:09:31.287308 26114 ts_tablet_manager.cc:1939] T a4936bd5a1974781b779ddce4d62eb80 P bfc1aeb863a54ffda5f0a9e0dd2104e9: Deleting consensus metadata
I20260929 03:09:31.288569 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 25816
I20260929 03:09:31.288683 25612 catalog_manager.cc:4841] TS bfc1aeb863a54ffda5f0a9e0dd2104e9 (127.7.209.66:39895): tablet a4936bd5a1974781b779ddce4d62eb80 (table test-table [id=ac8afc91dab24ee0bc327b2a0e1e32a2]) successfully deleted
I20260929 03:09:31.290530 26115 ts_tablet_manager.cc:1905] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:09:31.293480 26115 ts_tablet_manager.cc:1918] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:09:31.293576 26115 log.cc:1196] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/a4936bd5a1974781b779ddce4d62eb80
I20260929 03:09:31.293918 26115 ts_tablet_manager.cc:1939] T a4936bd5a1974781b779ddce4d62eb80 P c06414aa02854b978749363a69a9b963: Deleting consensus metadata
I20260929 03:09:31.295275 25612 catalog_manager.cc:4841] TS c06414aa02854b978749363a69a9b963 (127.7.209.67:36973): tablet a4936bd5a1974781b779ddce4d62eb80 (table test-table [id=ac8afc91dab24ee0bc327b2a0e1e32a2]) successfully deleted
I20260929 03:09:31.296098 26087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:31.295727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:35193 (local address 127.0.0.1:43744)
0929 03:09:31.295884 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:31.295991 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:35193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 03:09:31.296518 25611 catalog_manager.cc:4584] TS 1571c7b085484b67b6a842e91a3cc82f (127.7.209.65:35193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a4936bd5a1974781b779ddce4d62eb80: Network error: Client connection negotiation failed: client connection to 127.7.209.65:35193: connect: Connection refused (error 111)
I20260929 03:09:31.296653 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 25947
I20260929 03:09:31.302191 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 25594
2026-09-29T03:09:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1163 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:31 dist-test-slave-bdps krb5kdc[26119](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:31 dist-test-slave-bdps krb5kdc[26119](info): set up 2 sockets
Sep 29 03:09:31 dist-test-slave-bdps krb5kdc[26119](info): commencing operation
krb5kdc: starting...
W20260929 03:09:32.261821 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:32 dist-test-slave-bdps krb5kdc[26119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:09:32.384543 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:34103
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:34103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:32.522646 26132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:32.523012 26132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:32.523078 26132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:32.527283 26132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:32.527351 26132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:32.527380 26132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:32.527403 26132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:32.527432 26132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:32.532974 26132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:34103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:34103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:32.534368 26132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:32.535616 26132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:32.544936 26140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:32.544947 26138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:32.544947 26137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:32.545369 26132 server_base.cc:1034] running on GCE node
I20260929 03:09:32.546356 26132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:32.547647 26132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:32.548818 26132 hybrid_clock.cc:648] HybridClock initialized: now 1790651372548795 us; error 42 us; skew 500 ppm
Sep 29 03:09:32 dist-test-slave-bdps krb5kdc[26119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651372, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:32.551811 26132 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:09:32.552829 26132 webserver.cc:460] Webserver started at http://127.7.209.126:44393/ using document root <none> and password file <none>
I20260929 03:09:32.553514 26132 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:32.553583 26132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:32.553862 26132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:32.556360 26132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "5f15136bc2334bf6b2a7edded42ade76"
format_stamp: "Formatted at 2026-09-29 03:09:32 on dist-test-slave-bdps"
server_key: "85b18aab8b45be341f0a42f2594f43c8"
server_key_iv: "106d833f29d1924eb00ffb4f883b971e"
server_key_version: "encryptionkey@0"
I20260929 03:09:32.556994 26132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "5f15136bc2334bf6b2a7edded42ade76"
format_stamp: "Formatted at 2026-09-29 03:09:32 on dist-test-slave-bdps"
server_key: "85b18aab8b45be341f0a42f2594f43c8"
server_key_iv: "106d833f29d1924eb00ffb4f883b971e"
server_key_version: "encryptionkey@0"
I20260929 03:09:32.561417 26132 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260929 03:09:32.564100 26147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:32.565491 26132 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 03:09:32.565630 26132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "5f15136bc2334bf6b2a7edded42ade76"
format_stamp: "Formatted at 2026-09-29 03:09:32 on dist-test-slave-bdps"
server_key: "85b18aab8b45be341f0a42f2594f43c8"
server_key_iv: "106d833f29d1924eb00ffb4f883b971e"
server_key_version: "encryptionkey@0"
I20260929 03:09:32.565737 26132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:32.578778 26132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:32.581839 26132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:32.582024 26132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:32.589924 26132 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:34103
I20260929 03:09:32.589943 26199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:34103 every 8 connection(s)
I20260929 03:09:32.591127 26132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:32.594712 26200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:32.601071 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 26132
I20260929 03:09:32.601238 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:32.601400 26200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76: Bootstrap starting.
I20260929 03:09:32.601567 8005 external_mini_cluster.cc:1428] Setting key af9ba081a16f941e352068d8736569e2
I20260929 03:09:32.604327 26200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:32.605156 26200 log.cc:824] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:32.607353 26200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76: No bootstrap required, opened a new log
Sep 29 03:09:32 dist-test-slave-bdps krb5kdc[26119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:09:32.611897 26200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f15136bc2334bf6b2a7edded42ade76" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34103 } }
I20260929 03:09:32.612203 26200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:32.612275 26200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f15136bc2334bf6b2a7edded42ade76, State: Initialized, Role: FOLLOWER
I20260929 03:09:32.612793 26200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f15136bc2334bf6b2a7edded42ade76" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34103 } }
I20260929 03:09:32.612957 26200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:32.613029 26200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:32.613128 26200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:32.614127 26200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f15136bc2334bf6b2a7edded42ade76" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34103 } }
I20260929 03:09:32.614535 26200 leader_election.cc:302] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f15136bc2334bf6b2a7edded42ade76; no voters:
I20260929 03:09:32.614885 26200 leader_election.cc:288] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:32.615011 26205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:32.615233 26205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [term 1 LEADER]: Becoming Leader. State: Replica: 5f15136bc2334bf6b2a7edded42ade76, State: Running, Role: LEADER
I20260929 03:09:32.615576 26205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f15136bc2334bf6b2a7edded42ade76" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34103 } }
I20260929 03:09:32.616338 26200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:32.616873 26207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f15136bc2334bf6b2a7edded42ade76. Latest consensus state: current_term: 1 leader_uuid: "5f15136bc2334bf6b2a7edded42ade76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f15136bc2334bf6b2a7edded42ade76" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34103 } } }
I20260929 03:09:32.617008 26207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:32.617379 26206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f15136bc2334bf6b2a7edded42ade76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f15136bc2334bf6b2a7edded42ade76" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 34103 } } }
I20260929 03:09:32.617465 26206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:32.617862 26210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:32.618418 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:32.603182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42744 (local address 127.7.209.126:34103)
0929 03:09:32.603633 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:32.603651 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:32.603704 (+ 53us) server_negotiation.cc:407] Connection header received
0929 03:09:32.604399 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:32.604418 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:32.604841 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:32.605174 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:32.606176 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:32.607247 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:32.608503 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:32.609020 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:32.609150 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:32.612241 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:32.612247 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:32.612262 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:32.612298 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:32.614152 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:32.614633 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:32.614639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:32.614646 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:32.614808 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:32.615140 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:32.615144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:32.615146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:32.615795 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:32.616016 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:32.616812 (+ 796us) server_negotiation.cc:299] Negotiation successful
0929 03:09:32.617142 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":128,"threads_started":1}
I20260929 03:09:32.621443 26210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:32.627332 26210 catalog_manager.cc:1347] Generated new cluster ID: 97926e14ea3645fea059e3c76acbc33c
I20260929 03:09:32.627391 26210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:32.647078 26210 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:32.648792 26210 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:32.654306 26210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76: Generated new TSK 0
I20260929 03:09:32.655339 26210 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:09:32.686584 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34103
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:32.813725 26226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:32.814020 26226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:32.814065 26226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:32.818306 26226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:32.818362 26226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:32.818434 26226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:09:32.823766 26226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:32.825006 26226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:32.826278 26226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:32.837119 26231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:32.837146 26234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:32.837119 26232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:32.837224 26226 server_base.cc:1034] running on GCE node
I20260929 03:09:32.837787 26226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:32.838438 26226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:32.839587 26226 hybrid_clock.cc:648] HybridClock initialized: now 1790651372839567 us; error 36 us; skew 500 ppm
Sep 29 03:09:32 dist-test-slave-bdps krb5kdc[26119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651372, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:32.848335 26226 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:09:32.849267 26226 webserver.cc:460] Webserver started at http://127.7.209.65:37803/ using document root <none> and password file <none>
I20260929 03:09:32.849925 26226 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:32.849992 26226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:32.850214 26226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:32.852504 26226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "f2149ca065274c488b2c051d5897724f"
format_stamp: "Formatted at 2026-09-29 03:09:32 on dist-test-slave-bdps"
server_key: "595223a30a736da5c54d721d034e6c68"
server_key_iv: "4dd40d355aebcb82da1fa6f0c7b50155"
server_key_version: "encryptionkey@0"
I20260929 03:09:32.853108 26226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "f2149ca065274c488b2c051d5897724f"
format_stamp: "Formatted at 2026-09-29 03:09:32 on dist-test-slave-bdps"
server_key: "595223a30a736da5c54d721d034e6c68"
server_key_iv: "4dd40d355aebcb82da1fa6f0c7b50155"
server_key_version: "encryptionkey@0"
I20260929 03:09:32.857162 26226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 03:09:32.859755 26241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:32.861056 26226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 03:09:32.861155 26226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "f2149ca065274c488b2c051d5897724f"
format_stamp: "Formatted at 2026-09-29 03:09:32 on dist-test-slave-bdps"
server_key: "595223a30a736da5c54d721d034e6c68"
server_key_iv: "4dd40d355aebcb82da1fa6f0c7b50155"
server_key_version: "encryptionkey@0"
I20260929 03:09:32.861251 26226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:32.888134 26226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:32.891840 26226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:32.892023 26226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:32.892664 26226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:32.893643 26226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:32.893693 26226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:32.893736 26226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:32.893764 26226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:32.903264 26226 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:37001
I20260929 03:09:32.903442 26354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:37001 every 8 connection(s)
I20260929 03:09:32.904436 26226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:09:32.914083 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 26226
I20260929 03:09:32.914239 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:09:32.914584 8005 external_mini_cluster.cc:1428] Setting key 737809892059478fef67583729644642
Sep 29 03:09:32 dist-test-slave-bdps krb5kdc[26119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651372, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:09:32.920202 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:32.906677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:49027 (local address 127.7.209.126:34103)
0929 03:09:32.906858 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:32.906862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:32.907844 (+ 982us) server_negotiation.cc:407] Connection header received
0929 03:09:32.909389 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:32.909393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:32.909468 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:32.909553 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:32.911061 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:32.911663 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:32.912925 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:32.913184 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:32.913280 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:32.916268 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:32.916272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:32.916275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:32.916306 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:32.918031 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:32.918555 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:32.918558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:32.918560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:32.918663 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:32.918978 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:32.918982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:32.918984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:32.919186 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:32.919281 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:32.919849 (+ 568us) server_negotiation.cc:299] Negotiation successful
0929 03:09:32.920000 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 03:09:32.921275 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:32.907120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:34103 (local address 127.7.209.65:49027)
0929 03:09:32.907639 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:32.907704 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:32.909114 (+ 1410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:32.909814 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:32.909823 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:32.910279 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:32.910886 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:32.910903 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:32.911798 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:32.911801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:32.912792 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:32.912801 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:32.913352 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:32.913356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:32.913547 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:32.914267 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:09:32.914302 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:32.916079 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:32.918170 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:32.918175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:32.918189 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:32.918464 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:32.918745 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:32.918747 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:32.918749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:32.918885 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:32.919302 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:32.919309 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:32.919592 (+ 283us) client_negotiation.cc:769] Sending connection context
0929 03:09:32.919787 (+ 195us) client_negotiation.cc:240] Negotiation successful
0929 03:09:32.920179 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":136,"threads_started":1}
I20260929 03:09:32.922827 26355 heartbeater.cc:347] Connected to a master server at 127.7.209.126:34103
I20260929 03:09:32.923161 26355 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:32.923915 26355 heartbeater.cc:511] Master 127.7.209.126:34103 requested a full tablet report, sending...
I20260929 03:09:32.926039 26164 ts_manager.cc:194] Registered new tserver with Master: f2149ca065274c488b2c051d5897724f (127.7.209.65:37001)
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:09:32.927572 26164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:49027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:09:32.944407 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34103
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:33.064205 26360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:33.064513 26360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:33.064559 26360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:33.068699 26360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:33.068791 26360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:33.068854 26360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:09:33.075212 26360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:33.076678 26360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:33.078085 26360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:09:33.089752 26360 server_base.cc:1034] running on GCE node
W20260929 03:09:33.089754 26366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:33.089861 26365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:33.090030 26368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:33.090471 26360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:33.091212 26360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:33.092381 26360 hybrid_clock.cc:648] HybridClock initialized: now 1790651373092358 us; error 36 us; skew 500 ppm
Sep 29 03:09:33 dist-test-slave-bdps krb5kdc[26119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651373, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:33.095384 26360 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:09:33.096357 26360 webserver.cc:460] Webserver started at http://127.7.209.66:35201/ using document root <none> and password file <none>
I20260929 03:09:33.096974 26360 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:33.097036 26360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:33.097244 26360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:33.099622 26360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "d813d021dbbb44899626d5e9871586cf"
format_stamp: "Formatted at 2026-09-29 03:09:33 on dist-test-slave-bdps"
server_key: "8fa63f7e1c539c38dc00b2badf7be802"
server_key_iv: "f8205da7eb11cae3543fd74385a01238"
server_key_version: "encryptionkey@0"
I20260929 03:09:33.100224 26360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "d813d021dbbb44899626d5e9871586cf"
format_stamp: "Formatted at 2026-09-29 03:09:33 on dist-test-slave-bdps"
server_key: "8fa63f7e1c539c38dc00b2badf7be802"
server_key_iv: "f8205da7eb11cae3543fd74385a01238"
server_key_version: "encryptionkey@0"
I20260929 03:09:33.104414 26360 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:09:33.107112 26375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:33.108546 26360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:09:33.108664 26360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "d813d021dbbb44899626d5e9871586cf"
format_stamp: "Formatted at 2026-09-29 03:09:33 on dist-test-slave-bdps"
server_key: "8fa63f7e1c539c38dc00b2badf7be802"
server_key_iv: "f8205da7eb11cae3543fd74385a01238"
server_key_version: "encryptionkey@0"
I20260929 03:09:33.108770 26360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:33.132115 26360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:33.135401 26360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:33.135613 26360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:33.136401 26360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:33.137401 26360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:33.137462 26360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:33.137527 26360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:33.137558 26360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:33.147099 26360 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:34425
I20260929 03:09:33.147126 26488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:34425 every 8 connection(s)
I20260929 03:09:33.148106 26360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:09:33.150995 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 26360
I20260929 03:09:33.151151 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:09:33.151518 8005 external_mini_cluster.cc:1428] Setting key a58c15543679b612f62a9890f551c228
Sep 29 03:09:33 dist-test-slave-bdps krb5kdc[26119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651373, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:09:33.163245 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.150143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:54037 (local address 127.7.209.126:34103)
0929 03:09:33.150288 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:33.150292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:33.151326 (+ 1034us) server_negotiation.cc:407] Connection header received
0929 03:09:33.152832 (+ 1506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:33.152836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:33.152893 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:33.152971 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:33.154441 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.155010 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.156336 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.156670 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.156789 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.159550 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:33.159553 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:33.159555 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:33.159581 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:33.160959 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:33.161537 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:33.161540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:33.161542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:33.161638 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:33.162008 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:33.162010 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:33.162012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:33.162201 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:33.162284 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:33.162946 (+ 662us) server_negotiation.cc:299] Negotiation successful
0929 03:09:33.163080 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:09:33.164254 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.150580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:34103 (local address 127.7.209.66:54037)
0929 03:09:33.151143 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:33.151206 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:33.152613 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:33.153179 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:33.153188 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:33.153627 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:33.154300 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.154319 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.155132 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.155135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.156196 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.156207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.156818 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.156821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.156982 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.157595 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:09:33.157647 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:33.159371 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:33.161074 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:33.161080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:33.161100 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:33.161448 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:33.161718 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:33.161722 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:33.161724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:33.161923 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:33.162310 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:33.162317 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:33.162659 (+ 342us) client_negotiation.cc:769] Sending connection context
0929 03:09:33.162909 (+ 250us) client_negotiation.cc:240] Negotiation successful
0929 03:09:33.163265 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":180,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
I20260929 03:09:33.165670 26489 heartbeater.cc:347] Connected to a master server at 127.7.209.126:34103
I20260929 03:09:33.165987 26489 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:33.166687 26489 heartbeater.cc:511] Master 127.7.209.126:34103 requested a full tablet report, sending...
I20260929 03:09:33.168017 26164 ts_manager.cc:194] Registered new tserver with Master: d813d021dbbb44899626d5e9871586cf (127.7.209.66:34425)
I20260929 03:09:33.168856 26164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:54037
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:09:33.182018 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34103
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:34367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:33.306584 26494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:33.306855 26494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:33.306895 26494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:33.310971 26494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:33.311041 26494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:33.311102 26494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:09:33.316493 26494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:34367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:34103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:33.317759 26494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:33.318975 26494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:33.329267 26502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:33.329315 26500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:33.329282 26499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:33.329547 26494 server_base.cc:1034] running on GCE node
I20260929 03:09:33.329988 26494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:33.330639 26494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:33.331813 26494 hybrid_clock.cc:648] HybridClock initialized: now 1790651373331797 us; error 31 us; skew 500 ppm
Sep 29 03:09:33 dist-test-slave-bdps krb5kdc[26119](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651373, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:33.334645 26494 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:09:33.335541 26494 webserver.cc:460] Webserver started at http://127.7.209.67:37927/ using document root <none> and password file <none>
I20260929 03:09:33.336110 26494 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:33.336192 26494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:33.336406 26494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:33.338575 26494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "911b12f27a3b48d0b335c40ea92d4aac"
format_stamp: "Formatted at 2026-09-29 03:09:33 on dist-test-slave-bdps"
server_key: "fca756d8d0dc4816d1f49cc840629823"
server_key_iv: "f7ff05d5016948a6cc798cb47810b1a7"
server_key_version: "encryptionkey@0"
I20260929 03:09:33.339131 26494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "911b12f27a3b48d0b335c40ea92d4aac"
format_stamp: "Formatted at 2026-09-29 03:09:33 on dist-test-slave-bdps"
server_key: "fca756d8d0dc4816d1f49cc840629823"
server_key_iv: "f7ff05d5016948a6cc798cb47810b1a7"
server_key_version: "encryptionkey@0"
I20260929 03:09:33.342998 26494 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:09:33.345471 26509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:33.346794 26494 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 03:09:33.346904 26494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "911b12f27a3b48d0b335c40ea92d4aac"
format_stamp: "Formatted at 2026-09-29 03:09:33 on dist-test-slave-bdps"
server_key: "fca756d8d0dc4816d1f49cc840629823"
server_key_iv: "f7ff05d5016948a6cc798cb47810b1a7"
server_key_version: "encryptionkey@0"
I20260929 03:09:33.346999 26494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:33.363528 26494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:33.367018 26494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:33.367254 26494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:33.367973 26494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:33.368953 26494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:33.369016 26494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:33.369076 26494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:33.369105 26494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:33.379125 26494 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:43767
I20260929 03:09:33.379148 26622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:43767 every 8 connection(s)
I20260929 03:09:33.380267 26494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:09:33.390014 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 26494
I20260929 03:09:33.390198 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:09:33.390637 8005 external_mini_cluster.cc:1428] Setting key d68d7cf2faf6623cfbdeb6e26a48b209
Sep 29 03:09:33 dist-test-slave-bdps krb5kdc[26119](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651373, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:09:33.396081 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.382433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:49951 (local address 127.7.209.126:34103)
0929 03:09:33.382585 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:33.382589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:33.383642 (+ 1053us) server_negotiation.cc:407] Connection header received
0929 03:09:33.385156 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:33.385159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:33.385235 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:33.385321 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:33.386835 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.387433 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.388732 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.388985 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.389090 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.392401 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:33.392405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:33.392407 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:33.392440 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:33.394054 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:33.394541 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:33.394543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:33.394545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:33.394622 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:33.394913 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:33.394915 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:33.394916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:33.395110 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:33.395186 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:33.395731 (+ 545us) server_negotiation.cc:299] Negotiation successful
0929 03:09:33.395939 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 03:09:33.397095 26625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.382913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:34103 (local address 127.7.209.67:49951)
0929 03:09:33.383461 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:33.383513 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:33.384901 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:33.385545 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:33.385553 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:33.386045 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:33.386668 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.386685 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.387565 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.387568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.388601 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.388611 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.389153 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.389156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.389360 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.390147 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:09:33.390172 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:33.392229 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:33.394158 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:33.394163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:33.394180 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:33.394461 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:33.394699 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:33.394701 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:33.394702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:33.394848 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:33.395184 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:33.395190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:33.395483 (+ 293us) client_negotiation.cc:769] Sending connection context
0929 03:09:33.395661 (+ 178us) client_negotiation.cc:240] Negotiation successful
0929 03:09:33.396020 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":152,"threads_started":1}
I20260929 03:09:33.398355 26623 heartbeater.cc:347] Connected to a master server at 127.7.209.126:34103
I20260929 03:09:33.398622 26623 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:33.399173 26623 heartbeater.cc:511] Master 127.7.209.126:34103 requested a full tablet report, sending...
I20260929 03:09:33.400385 26164 ts_manager.cc:194] Registered new tserver with Master: 911b12f27a3b48d0b335c40ea92d4aac (127.7.209.67:43767)
I20260929 03:09:33.401026 26164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:49951
I20260929 03:09:33.404878 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:09:33.414491 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.407114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.7.209.126:34103)
0929 03:09:33.407261 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:33.407266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:33.407458 (+ 192us) server_negotiation.cc:407] Connection header received
0929 03:09:33.407560 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:33.407562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:33.407602 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:33.407677 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:33.408359 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.408893 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.409728 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.409945 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.410022 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.411512 (+ 1490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:33.411515 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:33.411518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:33.411541 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:33.412912 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:33.413359 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:33.413362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:33.413364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:33.413435 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:33.413684 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:33.413687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:33.413688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:33.413912 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:33.414033 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:33.414239 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 03:09:33.414321 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 03:09:33.417666 26164 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:09:33.421326 26164 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:09:33.437219 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.431635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42616 (local address 127.7.209.66:34425)
0929 03:09:33.432442 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:33.432454 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:33.432473 (+ 19us) server_negotiation.cc:407] Connection header received
0929 03:09:33.432543 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:33.432547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:33.432709 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:33.432879 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:33.433215 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.434220 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.435889 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.436537 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.436642 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.436824 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:33.436949 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 03:09:33.437053 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":209,"threads_started":1}
I20260929 03:09:33.437209 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.430875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:34425 (local address 127.0.0.1:42616)
0929 03:09:33.431920 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:33.431934 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:33.432029 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:33.432879 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:33.432886 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:33.432915 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:33.433087 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.433094 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.434342 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.434346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.435752 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.435763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.436658 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.436661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.436772 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.436810 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 03:09:33.436918 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 03:09:33.437045 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"mutex_wait_us":45,"thread_start_us":129,"threads_started":1}
I20260929 03:09:33.437532 26635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.430799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:43767 (local address 127.0.0.1:49952)
0929 03:09:33.431764 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:33.431828 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:33.431976 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:33.433074 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:33.433076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:33.433090 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:33.433184 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.433190 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.434903 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.434906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.435931 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.435941 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.437165 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.437169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.437267 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.437286 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 03:09:33.437344 (+ 58us) client_negotiation.cc:240] Negotiation successful
0929 03:09:33.437402 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"mutex_wait_us":43,"thread_start_us":90,"threads_started":1}
I20260929 03:09:33.437867 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.432225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34890 (local address 127.7.209.65:37001)
0929 03:09:33.432543 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:33.432550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:33.432573 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:09:33.432658 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:33.432664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:33.432849 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:33.433240 (+ 391us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:33.433391 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.434568 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.436023 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.436857 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.437300 (+ 443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.437528 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:33.437582 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 03:09:33.437718 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":111,"threads_started":1}
I20260929 03:09:33.438125 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.430973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49952 (local address 127.7.209.67:43767)
0929 03:09:33.432313 (+ 1340us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:33.432328 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:33.432354 (+ 26us) server_negotiation.cc:407] Connection header received
0929 03:09:33.432446 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:33.432454 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:33.432702 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:33.432880 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:33.433313 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.434586 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.436051 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.436857 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.436962 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.437111 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:33.437878 (+ 767us) server_negotiation.cc:299] Negotiation successful
0929 03:09:33.437983 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1207,"thread_start_us":160,"threads_started":1}
I20260929 03:09:33.438747 26634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.431679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:37001 (local address 127.0.0.1:34890)
0929 03:09:33.432112 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:33.432126 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:33.432202 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:33.432995 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:33.432998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:33.433011 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:33.433088 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.433094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.434687 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.434690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.435922 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.435930 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.436990 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.436994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.437076 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.437098 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 03:09:33.437146 (+ 48us) client_negotiation.cc:240] Negotiation successful
0929 03:09:33.437218 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":85,"thread_start_us":109,"threads_started":1}
I20260929 03:09:33.441530 26289 tablet_service.cc:1467] Processing CreateTablet for tablet 8b246537ca304612af841e9b2959b64b (DEFAULT_TABLE table=test-table [id=18b9683d8aa2447dafa1c9d7c12a71f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:33.441531 26423 tablet_service.cc:1467] Processing CreateTablet for tablet 8b246537ca304612af841e9b2959b64b (DEFAULT_TABLE table=test-table [id=18b9683d8aa2447dafa1c9d7c12a71f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:33.442894 26423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b246537ca304612af841e9b2959b64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:33.442894 26289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b246537ca304612af841e9b2959b64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:33.443781 26557 tablet_service.cc:1467] Processing CreateTablet for tablet 8b246537ca304612af841e9b2959b64b (DEFAULT_TABLE table=test-table [id=18b9683d8aa2447dafa1c9d7c12a71f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:33.444691 26557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b246537ca304612af841e9b2959b64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:33.450856 26641 tablet_bootstrap.cc:492] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf: Bootstrap starting.
I20260929 03:09:33.451295 26640 tablet_bootstrap.cc:492] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f: Bootstrap starting.
I20260929 03:09:33.452160 26642 tablet_bootstrap.cc:492] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac: Bootstrap starting.
I20260929 03:09:33.454181 26642 tablet_bootstrap.cc:654] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:33.454190 26640 tablet_bootstrap.cc:654] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:33.454646 26641 tablet_bootstrap.cc:654] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:33.455220 26642 log.cc:824] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:33.455193 26640 log.cc:824] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:33.455415 26641 log.cc:824] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:33.457186 26641 tablet_bootstrap.cc:492] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf: No bootstrap required, opened a new log
I20260929 03:09:33.457240 26640 tablet_bootstrap.cc:492] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f: No bootstrap required, opened a new log
I20260929 03:09:33.457240 26642 tablet_bootstrap.cc:492] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac: No bootstrap required, opened a new log
I20260929 03:09:33.457389 26641 ts_tablet_manager.cc:1397] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260929 03:09:33.457415 26642 ts_tablet_manager.cc:1397] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 03:09:33.457415 26640 ts_tablet_manager.cc:1397] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:09:33.460803 26641 raft_consensus.cc:348] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911b12f27a3b48d0b335c40ea92d4aac" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 43767 } } peers { permanent_uuid: "d813d021dbbb44899626d5e9871586cf" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34425 } } peers { permanent_uuid: "f2149ca065274c488b2c051d5897724f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37001 } }
I20260929 03:09:33.461056 26641 raft_consensus.cc:374] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:33.461118 26641 raft_consensus.cc:729] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d813d021dbbb44899626d5e9871586cf, State: Initialized, Role: FOLLOWER
I20260929 03:09:33.461290 26642 raft_consensus.cc:348] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911b12f27a3b48d0b335c40ea92d4aac" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 43767 } } peers { permanent_uuid: "d813d021dbbb44899626d5e9871586cf" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34425 } } peers { permanent_uuid: "f2149ca065274c488b2c051d5897724f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37001 } }
I20260929 03:09:33.461575 26642 raft_consensus.cc:374] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:33.461652 26642 raft_consensus.cc:729] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 911b12f27a3b48d0b335c40ea92d4aac, State: Initialized, Role: FOLLOWER
I20260929 03:09:33.461583 26641 consensus_queue.cc:260] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911b12f27a3b48d0b335c40ea92d4aac" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 43767 } } peers { permanent_uuid: "d813d021dbbb44899626d5e9871586cf" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34425 } } peers { permanent_uuid: "f2149ca065274c488b2c051d5897724f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37001 } }
I20260929 03:09:33.461817 26640 raft_consensus.cc:348] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911b12f27a3b48d0b335c40ea92d4aac" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 43767 } } peers { permanent_uuid: "d813d021dbbb44899626d5e9871586cf" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34425 } } peers { permanent_uuid: "f2149ca065274c488b2c051d5897724f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37001 } }
I20260929 03:09:33.462061 26640 raft_consensus.cc:374] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:33.462112 26640 raft_consensus.cc:729] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2149ca065274c488b2c051d5897724f, State: Initialized, Role: FOLLOWER
I20260929 03:09:33.462111 26642 consensus_queue.cc:260] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911b12f27a3b48d0b335c40ea92d4aac" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 43767 } } peers { permanent_uuid: "d813d021dbbb44899626d5e9871586cf" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34425 } } peers { permanent_uuid: "f2149ca065274c488b2c051d5897724f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37001 } }
I20260929 03:09:33.462483 26489 heartbeater.cc:503] Master 127.7.209.126:34103 was elected leader, sending a full tablet report...
I20260929 03:09:33.462714 26641 ts_tablet_manager.cc:1428] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260929 03:09:33.462782 26623 heartbeater.cc:503] Master 127.7.209.126:34103 was elected leader, sending a full tablet report...
I20260929 03:09:33.463128 26642 ts_tablet_manager.cc:1428] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260929 03:09:33.463519 26640 consensus_queue.cc:260] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911b12f27a3b48d0b335c40ea92d4aac" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 43767 } } peers { permanent_uuid: "d813d021dbbb44899626d5e9871586cf" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34425 } } peers { permanent_uuid: "f2149ca065274c488b2c051d5897724f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37001 } }
I20260929 03:09:33.464463 26640 ts_tablet_manager.cc:1428] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260929 03:09:33.464789 26355 heartbeater.cc:503] Master 127.7.209.126:34103 was elected leader, sending a full tablet report...
I20260929 03:09:33.628443 26648 raft_consensus.cc:482] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:09:33.628739 26648 raft_consensus.cc:504] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911b12f27a3b48d0b335c40ea92d4aac" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 43767 } } peers { permanent_uuid: "d813d021dbbb44899626d5e9871586cf" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34425 } } peers { permanent_uuid: "f2149ca065274c488b2c051d5897724f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37001 } }
I20260929 03:09:33.630332 26648 leader_election.cc:288] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 911b12f27a3b48d0b335c40ea92d4aac (127.7.209.67:43767), d813d021dbbb44899626d5e9871586cf (127.7.209.66:34425)
W20260929 03:09:33.631385 26624 tablet.cc:2369] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:09:33.634765 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.630813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:59867 (local address 127.7.209.67:43767)
0929 03:09:33.630967 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:33.630972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:33.631303 (+ 331us) server_negotiation.cc:407] Connection header received
0929 03:09:33.631498 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:33.631502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:33.631570 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:33.631658 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:33.631957 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.632608 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.633705 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.634180 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.634301 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.634359 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:33.634541 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 03:09:33.634616 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:09:33.634780 26650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.630698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:43767 (local address 127.7.209.65:59867)
0929 03:09:33.631176 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:33.631192 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:33.631358 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:33.631683 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:33.631685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:33.631703 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:33.631839 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.631844 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.632716 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.632719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.633579 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.633588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.634305 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.634309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.634406 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.634431 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 03:09:33.634508 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 03:09:33.634595 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":179,"threads_started":1}
I20260929 03:09:33.634830 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.630698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:34425 (local address 127.7.209.65:39873)
0929 03:09:33.631194 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:33.631206 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:33.631356 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:33.632023 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:33.632026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:33.632043 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:33.632147 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.632153 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.632879 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.632883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.633693 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:33.633704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.634428 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.634430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:33.634512 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.634529 (+ 17us) client_negotiation.cc:769] Sending connection context
0929 03:09:33.634595 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 03:09:33.634664 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":144,"threads_started":1}
I20260929 03:09:33.634829 26638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.630806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:39873 (local address 127.7.209.66:34425)
0929 03:09:33.630967 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:33.630972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:33.631321 (+ 349us) server_negotiation.cc:407] Connection header received
0929 03:09:33.631857 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:33.631861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:33.631922 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:33.632002 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:33.632261 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.632759 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.633825 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.634257 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.634367 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.634420 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:33.634627 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 03:09:33.634712 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 03:09:33.635485 26577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b246537ca304612af841e9b2959b64b" candidate_uuid: "f2149ca065274c488b2c051d5897724f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "911b12f27a3b48d0b335c40ea92d4aac" is_pre_election: true
I20260929 03:09:33.635555 26443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b246537ca304612af841e9b2959b64b" candidate_uuid: "f2149ca065274c488b2c051d5897724f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d813d021dbbb44899626d5e9871586cf" is_pre_election: true
I20260929 03:09:33.635842 26577 raft_consensus.cc:2454] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2149ca065274c488b2c051d5897724f in term 0.
I20260929 03:09:33.635874 26443 raft_consensus.cc:2454] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2149ca065274c488b2c051d5897724f in term 0.
I20260929 03:09:33.636374 26244 leader_election.cc:302] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d813d021dbbb44899626d5e9871586cf, f2149ca065274c488b2c051d5897724f; no voters:
I20260929 03:09:33.636680 26648 raft_consensus.cc:2781] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:09:33.636787 26648 raft_consensus.cc:482] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:09:33.636822 26648 raft_consensus.cc:3037] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:33.638094 26648 raft_consensus.cc:504] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911b12f27a3b48d0b335c40ea92d4aac" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 43767 } } peers { permanent_uuid: "d813d021dbbb44899626d5e9871586cf" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34425 } } peers { permanent_uuid: "f2149ca065274c488b2c051d5897724f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37001 } }
I20260929 03:09:33.638458 26648 leader_election.cc:288] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [CANDIDATE]: Term 1 election: Requested vote from peers 911b12f27a3b48d0b335c40ea92d4aac (127.7.209.67:43767), d813d021dbbb44899626d5e9871586cf (127.7.209.66:34425)
I20260929 03:09:33.638830 26443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b246537ca304612af841e9b2959b64b" candidate_uuid: "f2149ca065274c488b2c051d5897724f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d813d021dbbb44899626d5e9871586cf"
I20260929 03:09:33.638829 26577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b246537ca304612af841e9b2959b64b" candidate_uuid: "f2149ca065274c488b2c051d5897724f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "911b12f27a3b48d0b335c40ea92d4aac"
I20260929 03:09:33.638978 26577 raft_consensus.cc:3037] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:33.638978 26443 raft_consensus.cc:3037] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:33.640384 26443 raft_consensus.cc:2454] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2149ca065274c488b2c051d5897724f in term 1.
I20260929 03:09:33.640384 26577 raft_consensus.cc:2454] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2149ca065274c488b2c051d5897724f in term 1.
I20260929 03:09:33.640697 26243 leader_election.cc:302] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 911b12f27a3b48d0b335c40ea92d4aac, f2149ca065274c488b2c051d5897724f; no voters:
I20260929 03:09:33.640868 26648 raft_consensus.cc:2781] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:33.641114 26648 raft_consensus.cc:686] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 1 LEADER]: Becoming Leader. State: Replica: f2149ca065274c488b2c051d5897724f, State: Running, Role: LEADER
I20260929 03:09:33.641521 26648 consensus_queue.cc:237] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911b12f27a3b48d0b335c40ea92d4aac" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 43767 } } peers { permanent_uuid: "d813d021dbbb44899626d5e9871586cf" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34425 } } peers { permanent_uuid: "f2149ca065274c488b2c051d5897724f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37001 } }
I20260929 03:09:33.645291 26164 catalog_manager.cc:5495] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f reported cstate change: term changed from 0 to 1, leader changed from <none> to f2149ca065274c488b2c051d5897724f (127.7.209.65). New cstate: current_term: 1 leader_uuid: "f2149ca065274c488b2c051d5897724f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "911b12f27a3b48d0b335c40ea92d4aac" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 43767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d813d021dbbb44899626d5e9871586cf" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2149ca065274c488b2c051d5897724f" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 37001 } health_report { overall_health: HEALTHY } } }
W20260929 03:09:33.649266 26490 tablet.cc:2369] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 03:09:33.655891 26356 tablet.cc:2369] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:09:33.673534 26639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.669816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34892 (local address 127.7.209.65:37001)
0929 03:09:33.669942 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:33.669946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:33.669961 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:09:33.670108 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:33.670111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:33.670186 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:33.670292 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:09:33.670537 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.671114 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.672084 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.672294 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.672367 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.672641 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:09:33.673174 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:09:33.673289 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:33.673348 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 03:09:33.673391 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 03:09:33.680536 26577 raft_consensus.cc:1261] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac [term 1 FOLLOWER]: Refusing update from remote peer f2149ca065274c488b2c051d5897724f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:09:33.680536 26443 raft_consensus.cc:1261] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf [term 1 FOLLOWER]: Refusing update from remote peer f2149ca065274c488b2c051d5897724f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:09:33.681300 26651 consensus_queue.cc:1035] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [LEADER]: Connected to new peer: Peer: permanent_uuid: "911b12f27a3b48d0b335c40ea92d4aac" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 43767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:09:33.681484 26648 consensus_queue.cc:1035] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d813d021dbbb44899626d5e9871586cf" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 34425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:09:33.689112 26654 mvcc.cc:205] Tried to move back new op lower bound from 7334508026586980352 to 7334508026439004160. Current Snapshot: MvccSnapshot[applied={T|T < 7334508026586980352}]
I20260929 03:09:33.690599 26655 mvcc.cc:205] Tried to move back new op lower bound from 7334508026586980352 to 7334508026439004160. Current Snapshot: MvccSnapshot[applied={T|T < 7334508026586980352}]
I20260929 03:09:33.699347 26163 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:09:33.699517 26163 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:09:33.702013 26163 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f15136bc2334bf6b2a7edded42ade76: Sending DeleteTablet for 3 replicas of tablet 8b246537ca304612af841e9b2959b64b
I20260929 03:09:33.702826 26423 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b246537ca304612af841e9b2959b64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:33 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:42616
I20260929 03:09:33.702819 26557 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b246537ca304612af841e9b2959b64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:33 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:49952
I20260929 03:09:33.703147 26289 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b246537ca304612af841e9b2959b64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:33 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:34890
I20260929 03:09:33.703563 26666 tablet_replica.cc:330] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f: stopping tablet replica
I20260929 03:09:33.703562 26665 tablet_replica.cc:330] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf: stopping tablet replica
I20260929 03:09:33.703917 26665 raft_consensus.cc:2229] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:09:33.703917 26666 raft_consensus.cc:2229] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:09:33.704166 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 26226
I20260929 03:09:33.704259 26666 raft_consensus.cc:2258] T 8b246537ca304612af841e9b2959b64b P f2149ca065274c488b2c051d5897724f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:33.704429 26665 raft_consensus.cc:2258] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:33.705832 26664 tablet_replica.cc:330] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac: stopping tablet replica
I20260929 03:09:33.706489 26665 ts_tablet_manager.cc:1905] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:09:33.706688 26664 raft_consensus.cc:2229] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:09:33.706990 26664 raft_consensus.cc:2258] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:33.708606 26664 ts_tablet_manager.cc:1905] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:09:33.709380 26665 ts_tablet_manager.cc:1918] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:09:33.709481 26665 log.cc:1196] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/8b246537ca304612af841e9b2959b64b
I20260929 03:09:33.709846 26665 ts_tablet_manager.cc:1939] T 8b246537ca304612af841e9b2959b64b P d813d021dbbb44899626d5e9871586cf: Deleting consensus metadata
I20260929 03:09:33.711248 26150 catalog_manager.cc:4841] TS d813d021dbbb44899626d5e9871586cf (127.7.209.66:34425): tablet 8b246537ca304612af841e9b2959b64b (table test-table [id=18b9683d8aa2447dafa1c9d7c12a71f8]) successfully deleted
I20260929 03:09:33.711598 26664 ts_tablet_manager.cc:1918] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:09:33.711719 26664 log.cc:1196] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/8b246537ca304612af841e9b2959b64b
I20260929 03:09:33.712038 26664 ts_tablet_manager.cc:1939] T 8b246537ca304612af841e9b2959b64b P 911b12f27a3b48d0b335c40ea92d4aac: Deleting consensus metadata
I20260929 03:09:33.713439 26149 catalog_manager.cc:4841] TS 911b12f27a3b48d0b335c40ea92d4aac (127.7.209.67:43767): tablet 8b246537ca304612af841e9b2959b64b (table test-table [id=18b9683d8aa2447dafa1c9d7c12a71f8]) successfully deleted
W20260929 03:09:33.713999 26148 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:37001 (error 108)
I20260929 03:09:33.714334 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 26360
I20260929 03:09:33.715495 26635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.715155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:37001 (local address 127.0.0.1:34904)
0929 03:09:33.715303 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:33.715401 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:37001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260929 03:09:33.715658 26148 catalog_manager.cc:4584] TS f2149ca065274c488b2c051d5897724f (127.7.209.65:37001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b246537ca304612af841e9b2959b64b: Network error: Client connection negotiation failed: client connection to 127.7.209.65:37001: connect: Connection refused (error 111)
I20260929 03:09:33.721853 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 26494
I20260929 03:09:33.727428 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 26132
2026-09-29T03:09:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2426 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T03:09:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:33Z Disabled control of system clock
I20260929 03:09:33.759806 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:37561
--webserver_interface=127.7.209.126
--webserver_port=0
--builtin_ntp_servers=127.7.209.84:37401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:37561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 03:09:33.883764 26671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:33.884097 26671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:33.884135 26671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:33.888247 26671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:33.888316 26671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:33.888330 26671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:33.888345 26671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:33.888363 26671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:33.893165 26671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:37401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:37561
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:37561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:33.894484 26671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:33.896128 26671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:33.905718 26679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:33.905725 26676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:33.905858 26671 server_base.cc:1034] running on GCE node
W20260929 03:09:33.906118 26677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:33.906754 26671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:33.908062 26671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:33.909237 26671 hybrid_clock.cc:648] HybridClock initialized: now 1790651373909223 us; error 31 us; skew 500 ppm
I20260929 03:09:33.911420 26671 webserver.cc:460] Webserver started at http://127.7.209.126:37677/ using document root <none> and password file <none>
I20260929 03:09:33.912091 26671 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:33.912161 26671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:33.912410 26671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:33.914891 26671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "b82b30fa9cb340bfbde99c730edf1ac9"
format_stamp: "Formatted at 2026-09-29 03:09:33 on dist-test-slave-bdps"
server_key: "17bfb1eca96615b5a6e385d901617204"
server_key_iv: "b5c084e481fd73ea0d6b9343811092bd"
server_key_version: "encryptionkey@0"
I20260929 03:09:33.915501 26671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "b82b30fa9cb340bfbde99c730edf1ac9"
format_stamp: "Formatted at 2026-09-29 03:09:33 on dist-test-slave-bdps"
server_key: "17bfb1eca96615b5a6e385d901617204"
server_key_iv: "b5c084e481fd73ea0d6b9343811092bd"
server_key_version: "encryptionkey@0"
I20260929 03:09:33.919914 26671 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260929 03:09:33.922712 26685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:33.924149 26671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:33.924288 26671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "b82b30fa9cb340bfbde99c730edf1ac9"
format_stamp: "Formatted at 2026-09-29 03:09:33 on dist-test-slave-bdps"
server_key: "17bfb1eca96615b5a6e385d901617204"
server_key_iv: "b5c084e481fd73ea0d6b9343811092bd"
server_key_version: "encryptionkey@0"
I20260929 03:09:33.924391 26671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:33.952912 26671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:33.953830 26671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:33.954041 26671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:33.961890 26671 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:37561
I20260929 03:09:33.961901 26737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:37561 every 8 connection(s)
I20260929 03:09:33.962973 26671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:33.965726 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 26671
I20260929 03:09:33.965864 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:33.966221 8005 external_mini_cluster.cc:1428] Setting key 3d959bc6834c3f9f8cc9aff32b4b582e
I20260929 03:09:33.966856 26738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:33.973869 26738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9: Bootstrap starting.
I20260929 03:09:33.976210 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:33.967920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34856 (local address 127.7.209.126:37561)
0929 03:09:33.968634 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:33.968653 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:33.968696 (+ 43us) server_negotiation.cc:407] Connection header received
0929 03:09:33.969450 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:33.969468 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:33.969953 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:33.970328 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:33.970797 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.971830 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.972765 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:33.973273 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:33.973424 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:33.974387 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:33.974392 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:33.974405 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:33.974825 (+ 420us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:33.974885 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:33.974906 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:33.974999 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:33.975147 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 03:09:33.975422 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":162,"threads_started":1}
I20260929 03:09:33.976409 26738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:33.977206 26738 log.cc:824] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:33.979317 26738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9: No bootstrap required, opened a new log
I20260929 03:09:33.983014 26738 raft_consensus.cc:348] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82b30fa9cb340bfbde99c730edf1ac9" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37561 } }
I20260929 03:09:33.983229 26738 raft_consensus.cc:374] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:33.983273 26738 raft_consensus.cc:729] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b82b30fa9cb340bfbde99c730edf1ac9, State: Initialized, Role: FOLLOWER
I20260929 03:09:33.983651 26738 consensus_queue.cc:260] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82b30fa9cb340bfbde99c730edf1ac9" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37561 } }
I20260929 03:09:33.983777 26738 raft_consensus.cc:388] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:33.983836 26738 raft_consensus.cc:482] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:33.983909 26738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:33.984675 26738 raft_consensus.cc:504] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82b30fa9cb340bfbde99c730edf1ac9" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37561 } }
I20260929 03:09:33.985038 26738 leader_election.cc:302] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b82b30fa9cb340bfbde99c730edf1ac9; no voters:
I20260929 03:09:33.985355 26738 leader_election.cc:288] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:33.985433 26743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:33.985723 26743 raft_consensus.cc:686] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [term 1 LEADER]: Becoming Leader. State: Replica: b82b30fa9cb340bfbde99c730edf1ac9, State: Running, Role: LEADER
I20260929 03:09:33.986023 26743 consensus_queue.cc:237] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82b30fa9cb340bfbde99c730edf1ac9" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37561 } }
I20260929 03:09:33.986575 26738 sys_catalog.cc:564] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:33.987593 26744 sys_catalog.cc:455] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b82b30fa9cb340bfbde99c730edf1ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82b30fa9cb340bfbde99c730edf1ac9" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37561 } } }
I20260929 03:09:33.987737 26744 sys_catalog.cc:458] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:33.988059 26745 sys_catalog.cc:455] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b82b30fa9cb340bfbde99c730edf1ac9. Latest consensus state: current_term: 1 leader_uuid: "b82b30fa9cb340bfbde99c730edf1ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82b30fa9cb340bfbde99c730edf1ac9" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37561 } } }
I20260929 03:09:33.988166 26752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:33.988171 26745 sys_catalog.cc:458] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:33.991291 26752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:33.997236 26752 catalog_manager.cc:1347] Generated new cluster ID: b67554277adf4456a6b06f683486fdc9
I20260929 03:09:33.997316 26752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:34.006883 26752 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:34.008081 26752 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:34.011572 26752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9: Generated new TSK 0
I20260929 03:09:34.012229 26752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:34.020574 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:37561
--builtin_ntp_servers=127.7.209.84:37401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 03:09:34.152056 26762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:34.152415 26762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:34.152462 26762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:34.158090 26762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:34.158159 26762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:34.158237 26762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:09:34.165021 26762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:37401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:37561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:34.166448 26762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:34.168278 26762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:34.179102 26768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:34.179147 26770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:34.179133 26767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:34.179538 26762 server_base.cc:1034] running on GCE node
I20260929 03:09:34.179976 26762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:34.180706 26762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:34.181859 26762 hybrid_clock.cc:648] HybridClock initialized: now 1790651374181837 us; error 36 us; skew 500 ppm
I20260929 03:09:34.183974 26762 webserver.cc:460] Webserver started at http://127.7.209.65:46847/ using document root <none> and password file <none>
I20260929 03:09:34.184643 26762 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:34.184717 26762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:34.184957 26762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:34.187498 26762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "50d064de2baf4a129fa3fb9ec6be2792"
format_stamp: "Formatted at 2026-09-29 03:09:34 on dist-test-slave-bdps"
server_key: "06936d405b4c9869e9fd26fcafdccbcc"
server_key_iv: "7530f789db1fc2bbc9f5411becc51f0f"
server_key_version: "encryptionkey@0"
I20260929 03:09:34.188165 26762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "50d064de2baf4a129fa3fb9ec6be2792"
format_stamp: "Formatted at 2026-09-29 03:09:34 on dist-test-slave-bdps"
server_key: "06936d405b4c9869e9fd26fcafdccbcc"
server_key_iv: "7530f789db1fc2bbc9f5411becc51f0f"
server_key_version: "encryptionkey@0"
I20260929 03:09:34.192649 26762 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 03:09:34.195465 26776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:34.196872 26762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:09:34.197000 26762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "50d064de2baf4a129fa3fb9ec6be2792"
format_stamp: "Formatted at 2026-09-29 03:09:34 on dist-test-slave-bdps"
server_key: "06936d405b4c9869e9fd26fcafdccbcc"
server_key_iv: "7530f789db1fc2bbc9f5411becc51f0f"
server_key_version: "encryptionkey@0"
I20260929 03:09:34.197108 26762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:34.227559 26762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:34.228482 26762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:34.228679 26762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:34.229405 26762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:34.230597 26762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:34.230649 26762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:34.230707 26762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:34.230733 26762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:34.240731 26762 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:44233
I20260929 03:09:34.240756 26889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:44233 every 8 connection(s)
I20260929 03:09:34.241813 26762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:09:34.243119 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 26762
I20260929 03:09:34.243259 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:09:34.243609 8005 external_mini_cluster.cc:1428] Setting key 2cb9476a7166b243c3d70cd685f6e1e6
I20260929 03:09:34.245898 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:37561
--builtin_ntp_servers=127.7.209.84:37401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 03:09:34.252866 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.244007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:40223 (local address 127.7.209.126:37561)
0929 03:09:34.244167 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:34.244172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:34.245533 (+ 1361us) server_negotiation.cc:407] Connection header received
0929 03:09:34.247093 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:34.247097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:34.247159 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:34.247239 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:34.248526 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.249096 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.250752 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.250987 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.251067 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.252372 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:34.252375 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:34.252377 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:34.252388 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:34.252411 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:34.252419 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:34.252472 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:34.252640 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 03:09:34.252708 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:09:34.254257 26892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.244542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:37561 (local address 127.7.209.65:40223)
0929 03:09:34.245386 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:34.245432 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:34.246879 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:34.247438 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:34.247445 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:34.247565 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:34.248356 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.248375 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.249211 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.249215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.250629 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.250640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.251138 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.251143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.251367 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.252051 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:09:34.252054 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:34.252118 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:09:34.252209 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:34.252492 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:34.252500 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:34.252506 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 03:09:34.252651 (+ 145us) client_negotiation.cc:240] Negotiation successful
0929 03:09:34.252995 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":135,"threads_started":1}
I20260929 03:09:34.255535 26890 heartbeater.cc:347] Connected to a master server at 127.7.209.126:37561
I20260929 03:09:34.255787 26890 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:34.256362 26890 heartbeater.cc:511] Master 127.7.209.126:37561 requested a full tablet report, sending...
I20260929 03:09:34.258074 26702 ts_manager.cc:194] Registered new tserver with Master: 50d064de2baf4a129fa3fb9ec6be2792 (127.7.209.65:44233)
I20260929 03:09:34.259399 26702 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.209.65:40223
W20260929 03:09:34.370447 26893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:34.370779 26893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:34.370821 26893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:34.375005 26893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:34.375062 26893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:34.375118 26893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:09:34.379881 26893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:37401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:37561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:34.381171 26893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:34.382818 26893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:34.393884 26899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:34.393886 26898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:34.394105 26893 server_base.cc:1034] running on GCE node
W20260929 03:09:34.393884 26901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:34.394735 26893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:34.395548 26893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:34.396716 26893 hybrid_clock.cc:648] HybridClock initialized: now 1790651374396693 us; error 36 us; skew 500 ppm
I20260929 03:09:34.398859 26893 webserver.cc:460] Webserver started at http://127.7.209.66:36287/ using document root <none> and password file <none>
I20260929 03:09:34.399544 26893 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:34.399616 26893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:34.399845 26893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:34.402452 26893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "b8d8aad6ce2d4132ab25f1126999ab29"
format_stamp: "Formatted at 2026-09-29 03:09:34 on dist-test-slave-bdps"
server_key: "1048eede01b748148b3e0a4f0b2966fd"
server_key_iv: "f1b59a35aeee897136b7ebcea71f660f"
server_key_version: "encryptionkey@0"
I20260929 03:09:34.403080 26893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "b8d8aad6ce2d4132ab25f1126999ab29"
format_stamp: "Formatted at 2026-09-29 03:09:34 on dist-test-slave-bdps"
server_key: "1048eede01b748148b3e0a4f0b2966fd"
server_key_iv: "f1b59a35aeee897136b7ebcea71f660f"
server_key_version: "encryptionkey@0"
I20260929 03:09:34.407305 26893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:09:34.409951 26907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:34.411305 26893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 03:09:34.411417 26893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "b8d8aad6ce2d4132ab25f1126999ab29"
format_stamp: "Formatted at 2026-09-29 03:09:34 on dist-test-slave-bdps"
server_key: "1048eede01b748148b3e0a4f0b2966fd"
server_key_iv: "f1b59a35aeee897136b7ebcea71f660f"
server_key_version: "encryptionkey@0"
I20260929 03:09:34.411515 26893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:34.434752 26893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:34.435770 26893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:34.436009 26893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:34.436801 26893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:34.437978 26893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:34.438033 26893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:34.438090 26893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:34.438117 26893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:34.447717 26893 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:40341
I20260929 03:09:34.447764 27020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:40341 every 8 connection(s)
I20260929 03:09:34.448757 26893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:09:34.454428 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 26893
I20260929 03:09:34.454555 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:09:34.454933 8005 external_mini_cluster.cc:1428] Setting key 3a62c4f42b9d623ea114206521034cd7
I20260929 03:09:34.457269 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:37561
--builtin_ntp_servers=127.7.209.84:37401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 03:09:34.459398 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.450787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:47099 (local address 127.7.209.126:37561)
0929 03:09:34.450969 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:34.450974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:34.452154 (+ 1180us) server_negotiation.cc:407] Connection header received
0929 03:09:34.453677 (+ 1523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:34.453680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:34.453740 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:34.453808 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:34.455060 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.455714 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.457221 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.457514 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.457596 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.458938 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:34.458941 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:34.458943 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:34.458955 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:34.458978 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:34.458986 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:34.459044 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:34.459199 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 03:09:34.459256 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 03:09:34.460554 27023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.451265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:37561 (local address 127.7.209.66:47099)
0929 03:09:34.451993 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:34.452038 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:34.453418 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:34.453986 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:34.453993 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:34.454095 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:34.454892 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.454914 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.455845 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.455848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.457095 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.457107 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.457712 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.457716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.457926 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.458634 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:09:34.458636 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:34.458698 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:09:34.458786 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:34.459060 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:34.459067 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:34.459072 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 03:09:34.459208 (+ 136us) client_negotiation.cc:240] Negotiation successful
0929 03:09:34.459705 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":120,"threads_started":1}
I20260929 03:09:34.461709 27021 heartbeater.cc:347] Connected to a master server at 127.7.209.126:37561
I20260929 03:09:34.461961 27021 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:34.462567 27021 heartbeater.cc:511] Master 127.7.209.126:37561 requested a full tablet report, sending...
I20260929 03:09:34.463698 26702 ts_manager.cc:194] Registered new tserver with Master: b8d8aad6ce2d4132ab25f1126999ab29 (127.7.209.66:40341)
I20260929 03:09:34.464577 26702 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.209.66:47099
W20260929 03:09:34.582557 27024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:34.582839 27024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:34.582875 27024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:34.586825 27024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:34.586871 27024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:34.586922 27024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:09:34.591598 27024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:37401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:37561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:34.592810 27024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:34.594429 27024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:34.605073 27029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:34.605139 27024 server_base.cc:1034] running on GCE node
W20260929 03:09:34.605089 27030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:34.605073 27032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:34.605755 27024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:34.606427 27024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:34.607580 27024 hybrid_clock.cc:648] HybridClock initialized: now 1790651374607559 us; error 33 us; skew 500 ppm
I20260929 03:09:34.609513 27024 webserver.cc:460] Webserver started at http://127.7.209.67:33837/ using document root <none> and password file <none>
I20260929 03:09:34.610168 27024 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:34.610234 27024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:34.610456 27024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:34.612754 27024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "ac9ba2e0575049d7a0bfe53c04902fd7"
format_stamp: "Formatted at 2026-09-29 03:09:34 on dist-test-slave-bdps"
server_key: "0be6cbc6b11524898d120572cc910d5d"
server_key_iv: "6da2a97cae85457e52ef5f487a5f5a52"
server_key_version: "encryptionkey@0"
I20260929 03:09:34.613334 27024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "ac9ba2e0575049d7a0bfe53c04902fd7"
format_stamp: "Formatted at 2026-09-29 03:09:34 on dist-test-slave-bdps"
server_key: "0be6cbc6b11524898d120572cc910d5d"
server_key_iv: "6da2a97cae85457e52ef5f487a5f5a52"
server_key_version: "encryptionkey@0"
I20260929 03:09:34.617383 27024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 03:09:34.619978 27038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:34.621322 27024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 03:09:34.621428 27024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "ac9ba2e0575049d7a0bfe53c04902fd7"
format_stamp: "Formatted at 2026-09-29 03:09:34 on dist-test-slave-bdps"
server_key: "0be6cbc6b11524898d120572cc910d5d"
server_key_iv: "6da2a97cae85457e52ef5f487a5f5a52"
server_key_version: "encryptionkey@0"
I20260929 03:09:34.621531 27024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:34.648941 27024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:34.649958 27024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:34.650184 27024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:34.650925 27024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:34.652055 27024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:34.652105 27024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:34.652148 27024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:34.652168 27024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:34.661844 27024 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:42523
I20260929 03:09:34.661873 27151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:42523 every 8 connection(s)
I20260929 03:09:34.662905 27024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:09:34.665520 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 27024
I20260929 03:09:34.665647 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:09:34.666051 8005 external_mini_cluster.cc:1428] Setting key 21cce1ec9b3f0ea3a7382f58e6bb2777
I20260929 03:09:34.673465 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.664919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:51509 (local address 127.7.209.126:37561)
0929 03:09:34.665104 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:34.665109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:34.666598 (+ 1489us) server_negotiation.cc:407] Connection header received
0929 03:09:34.668457 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:34.668460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:34.668515 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:34.668587 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:34.669656 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.670244 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.671511 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.671757 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.671841 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.673017 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:34.673021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:34.673023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:34.673037 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:34.673060 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:34.673068 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:34.673125 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:34.673254 (+ 129us) server_negotiation.cc:299] Negotiation successful
0929 03:09:34.673316 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 03:09:34.674537 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.665449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:37561 (local address 127.7.209.67:51509)
0929 03:09:34.666395 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:34.666462 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:34.668205 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:34.668780 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:34.668787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:34.668894 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:34.669509 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.669521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.670353 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.670355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.671392 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.671402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.671891 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.671894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.672055 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.672729 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:09:34.672731 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:34.672785 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:09:34.672866 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:34.673133 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:34.673139 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:34.673143 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 03:09:34.673256 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 03:09:34.673535 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":194,"threads_started":1}
I20260929 03:09:34.675772 27152 heartbeater.cc:347] Connected to a master server at 127.7.209.126:37561
I20260929 03:09:34.676030 27152 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:34.676643 27152 heartbeater.cc:511] Master 127.7.209.126:37561 requested a full tablet report, sending...
I20260929 03:09:34.677803 26702 ts_manager.cc:194] Registered new tserver with Master: ac9ba2e0575049d7a0bfe53c04902fd7 (127.7.209.67:42523)
I20260929 03:09:34.678442 26702 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.209.67:51509
I20260929 03:09:34.680274 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:09:34.686604 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.682586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34860 (local address 127.7.209.126:37561)
0929 03:09:34.682719 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:34.682723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:34.682902 (+ 179us) server_negotiation.cc:407] Connection header received
0929 03:09:34.683021 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:34.683024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:34.683072 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:34.683138 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:34.683440 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.683984 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.684882 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.685111 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.685197 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.686110 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:34.686114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:34.686117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:34.686130 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:34.686151 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:34.686159 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:34.686274 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:34.686415 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 03:09:34.686453 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:09:34.689525 26702 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:09:34.693046 26702 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:09:34.708515 27164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.702479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:42523 (local address 127.0.0.1:36234)
0929 03:09:34.703156 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:34.703208 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:34.703349 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:34.704292 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:34.704299 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:34.704324 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:34.704483 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.704489 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.705853 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.705856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.707071 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.707080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.707940 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.707944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.708057 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.708101 (+ 44us) client_negotiation.cc:769] Sending connection context
0929 03:09:34.708217 (+ 116us) client_negotiation.cc:240] Negotiation successful
0929 03:09:34.708351 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":80,"threads_started":1}
I20260929 03:09:34.708549 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.702656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.7.209.67:42523)
0929 03:09:34.703683 (+ 1027us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:34.703698 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:34.703727 (+ 29us) server_negotiation.cc:407] Connection header received
0929 03:09:34.703836 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:34.703842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:34.704106 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:34.704658 (+ 552us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:34.704830 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.705741 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.707190 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.707781 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.707923 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.708154 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:34.708249 (+ 95us) server_negotiation.cc:299] Negotiation successful
0929 03:09:34.708358 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":903,"thread_start_us":105,"threads_started":1}
I20260929 03:09:34.709227 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.702540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:40341 (local address 127.0.0.1:42554)
0929 03:09:34.703595 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:34.703609 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:34.703687 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:34.704686 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:34.704689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:34.704708 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:34.704813 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.704819 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.706311 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.706315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.707737 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.707751 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.708831 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.708834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.708921 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.708945 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 03:09:34.709006 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 03:09:34.709082 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"mutex_wait_us":146,"thread_start_us":125,"threads_started":1}
I20260929 03:09:34.709326 27166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.703219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42554 (local address 127.7.209.66:40341)
0929 03:09:34.704191 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:34.704208 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:34.704246 (+ 38us) server_negotiation.cc:407] Connection header received
0929 03:09:34.704331 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:34.704336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:34.704534 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:34.704680 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:34.704923 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.706167 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.707881 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.708670 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.708822 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.708999 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:34.709053 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 03:09:34.709153 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":850,"thread_start_us":127,"threads_started":1}
I20260929 03:09:34.709776 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.703999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39196 (local address 127.7.209.65:44233)
0929 03:09:34.704308 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:34.704314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:34.704337 (+ 23us) server_negotiation.cc:407] Connection header received
0929 03:09:34.704408 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:34.704412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:34.704553 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:34.704699 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:34.705080 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.706255 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.707869 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.708676 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.708779 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.708979 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:34.709449 (+ 470us) server_negotiation.cc:299] Negotiation successful
0929 03:09:34.709567 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":103,"threads_started":1}
I20260929 03:09:34.710021 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.703625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:44233 (local address 127.0.0.1:39196)
0929 03:09:34.703860 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:34.703871 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:34.703946 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:34.704846 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:34.704849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:34.704867 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:34.704950 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.704955 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.706375 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.706377 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.707737 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.707748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.709037 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.709040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.709115 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.709136 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 03:09:34.709193 (+ 57us) client_negotiation.cc:240] Negotiation successful
0929 03:09:34.709271 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":46,"thread_start_us":80,"threads_started":1}
I20260929 03:09:34.712710 26955 tablet_service.cc:1467] Processing CreateTablet for tablet 88d6c8060b9a484b9e5d01faaa12ca44 (DEFAULT_TABLE table=test-table [id=6ba5ecf12b804dbfa3bd9873881aa047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:34.712713 26824 tablet_service.cc:1467] Processing CreateTablet for tablet 88d6c8060b9a484b9e5d01faaa12ca44 (DEFAULT_TABLE table=test-table [id=6ba5ecf12b804dbfa3bd9873881aa047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:34.712715 27086 tablet_service.cc:1467] Processing CreateTablet for tablet 88d6c8060b9a484b9e5d01faaa12ca44 (DEFAULT_TABLE table=test-table [id=6ba5ecf12b804dbfa3bd9873881aa047]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:34.713937 26824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88d6c8060b9a484b9e5d01faaa12ca44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:34.713937 27086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88d6c8060b9a484b9e5d01faaa12ca44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:34.713940 26955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88d6c8060b9a484b9e5d01faaa12ca44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:34.721254 27169 tablet_bootstrap.cc:492] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29: Bootstrap starting.
I20260929 03:09:34.722657 27170 tablet_bootstrap.cc:492] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792: Bootstrap starting.
I20260929 03:09:34.722838 27171 tablet_bootstrap.cc:492] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7: Bootstrap starting.
I20260929 03:09:34.723874 27169 tablet_bootstrap.cc:654] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:34.724850 27169 log.cc:824] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:34.724997 27171 tablet_bootstrap.cc:654] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:34.725018 27170 tablet_bootstrap.cc:654] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:34.725983 27171 log.cc:824] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:34.725982 27170 log.cc:824] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:34.727092 27169 tablet_bootstrap.cc:492] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29: No bootstrap required, opened a new log
I20260929 03:09:34.727312 27169 ts_tablet_manager.cc:1397] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:09:34.728083 27170 tablet_bootstrap.cc:492] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792: No bootstrap required, opened a new log
I20260929 03:09:34.728299 27170 ts_tablet_manager.cc:1397] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:09:34.728482 27171 tablet_bootstrap.cc:492] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7: No bootstrap required, opened a new log
I20260929 03:09:34.728667 27171 ts_tablet_manager.cc:1397] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:09:34.731175 27169 raft_consensus.cc:348] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42523 } } peers { permanent_uuid: "b8d8aad6ce2d4132ab25f1126999ab29" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40341 } } peers { permanent_uuid: "50d064de2baf4a129fa3fb9ec6be2792" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44233 } }
I20260929 03:09:34.731489 27169 raft_consensus.cc:374] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:34.731549 27169 raft_consensus.cc:729] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8d8aad6ce2d4132ab25f1126999ab29, State: Initialized, Role: FOLLOWER
I20260929 03:09:34.732048 27169 consensus_queue.cc:260] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42523 } } peers { permanent_uuid: "b8d8aad6ce2d4132ab25f1126999ab29" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40341 } } peers { permanent_uuid: "50d064de2baf4a129fa3fb9ec6be2792" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44233 } }
I20260929 03:09:34.732770 27021 heartbeater.cc:503] Master 127.7.209.126:37561 was elected leader, sending a full tablet report...
I20260929 03:09:34.732757 27170 raft_consensus.cc:348] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42523 } } peers { permanent_uuid: "b8d8aad6ce2d4132ab25f1126999ab29" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40341 } } peers { permanent_uuid: "50d064de2baf4a129fa3fb9ec6be2792" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44233 } }
I20260929 03:09:34.733022 27170 raft_consensus.cc:374] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:34.733076 27170 raft_consensus.cc:729] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d064de2baf4a129fa3fb9ec6be2792, State: Initialized, Role: FOLLOWER
I20260929 03:09:34.733098 27169 ts_tablet_manager.cc:1428] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 03:09:34.733381 27171 raft_consensus.cc:348] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42523 } } peers { permanent_uuid: "b8d8aad6ce2d4132ab25f1126999ab29" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40341 } } peers { permanent_uuid: "50d064de2baf4a129fa3fb9ec6be2792" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44233 } }
I20260929 03:09:34.733584 27171 raft_consensus.cc:374] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:34.733570 27170 consensus_queue.cc:260] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42523 } } peers { permanent_uuid: "b8d8aad6ce2d4132ab25f1126999ab29" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40341 } } peers { permanent_uuid: "50d064de2baf4a129fa3fb9ec6be2792" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44233 } }
I20260929 03:09:34.733673 27171 raft_consensus.cc:729] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac9ba2e0575049d7a0bfe53c04902fd7, State: Initialized, Role: FOLLOWER
I20260929 03:09:34.734351 26890 heartbeater.cc:503] Master 127.7.209.126:37561 was elected leader, sending a full tablet report...
I20260929 03:09:34.734675 27170 ts_tablet_manager.cc:1428] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 03:09:34.734097 27171 consensus_queue.cc:260] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42523 } } peers { permanent_uuid: "b8d8aad6ce2d4132ab25f1126999ab29" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40341 } } peers { permanent_uuid: "50d064de2baf4a129fa3fb9ec6be2792" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44233 } }
I20260929 03:09:34.735831 27152 heartbeater.cc:503] Master 127.7.209.126:37561 was elected leader, sending a full tablet report...
I20260929 03:09:34.736183 27171 ts_tablet_manager.cc:1428] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260929 03:09:34.742969 26891 tablet.cc:2369] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:09:34.810642 27176 raft_consensus.cc:482] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:09:34.810914 27176 raft_consensus.cc:504] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42523 } } peers { permanent_uuid: "b8d8aad6ce2d4132ab25f1126999ab29" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40341 } } peers { permanent_uuid: "50d064de2baf4a129fa3fb9ec6be2792" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44233 } }
I20260929 03:09:34.812352 27176 leader_election.cc:288] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac9ba2e0575049d7a0bfe53c04902fd7 (127.7.209.67:42523), b8d8aad6ce2d4132ab25f1126999ab29 (127.7.209.66:40341)
I20260929 03:09:34.817766 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.812737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:58389 (local address 127.7.209.67:42523)
0929 03:09:34.812908 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:34.812913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:34.813169 (+ 256us) server_negotiation.cc:407] Connection header received
0929 03:09:34.813325 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:34.813328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:34.813388 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:34.813477 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:34.813923 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.814758 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.816493 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.817114 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.817234 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.817292 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:34.817522 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 03:09:34.817593 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:09:34.817766 27166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.812851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:54615 (local address 127.7.209.66:40341)
0929 03:09:34.812973 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:34.812977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:34.813133 (+ 156us) server_negotiation.cc:407] Connection header received
0929 03:09:34.813332 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:34.813335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:34.813388 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:34.813480 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:34.813938 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.814743 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.816494 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.817111 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.817233 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.817292 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:34.817519 (+ 227us) server_negotiation.cc:299] Negotiation successful
0929 03:09:34.817596 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 03:09:34.817830 27178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.812629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:40341 (local address 127.7.209.65:54615)
0929 03:09:34.813030 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:34.813048 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:34.813210 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:34.813496 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:34.813499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:34.813527 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:34.813811 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.813820 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.814844 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.814847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.816375 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.816387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.817240 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.817244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.817396 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.817431 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 03:09:34.817496 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 03:09:34.817597 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":187,"spinlock_wait_cycles":102528,"thread_start_us":119,"threads_started":1}
I20260929 03:09:34.817830 27179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.812629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:42523 (local address 127.7.209.65:58389)
0929 03:09:34.813076 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:34.813088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:34.813210 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:34.813496 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:34.813499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:34.813527 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:34.813815 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.813821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.814844 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.814847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.816375 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:34.816387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.817233 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.817236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:34.817373 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.817419 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 03:09:34.817493 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 03:09:34.817597 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":192,"spinlock_wait_cycles":9472,"thread_start_us":132,"threads_started":1}
I20260929 03:09:34.818575 26975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d6c8060b9a484b9e5d01faaa12ca44" candidate_uuid: "50d064de2baf4a129fa3fb9ec6be2792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d8aad6ce2d4132ab25f1126999ab29" is_pre_election: true
I20260929 03:09:34.818574 27106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d6c8060b9a484b9e5d01faaa12ca44" candidate_uuid: "50d064de2baf4a129fa3fb9ec6be2792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7" is_pre_election: true
I20260929 03:09:34.818943 27106 raft_consensus.cc:2454] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d064de2baf4a129fa3fb9ec6be2792 in term 0.
I20260929 03:09:34.818943 26975 raft_consensus.cc:2454] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d064de2baf4a129fa3fb9ec6be2792 in term 0.
I20260929 03:09:34.819448 26777 leader_election.cc:302] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d064de2baf4a129fa3fb9ec6be2792, ac9ba2e0575049d7a0bfe53c04902fd7; no voters:
I20260929 03:09:34.819738 27176 raft_consensus.cc:2781] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:09:34.819832 27176 raft_consensus.cc:482] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:09:34.819880 27176 raft_consensus.cc:3037] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:34.821123 27176 raft_consensus.cc:504] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42523 } } peers { permanent_uuid: "b8d8aad6ce2d4132ab25f1126999ab29" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40341 } } peers { permanent_uuid: "50d064de2baf4a129fa3fb9ec6be2792" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44233 } }
I20260929 03:09:34.821456 27176 leader_election.cc:288] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [CANDIDATE]: Term 1 election: Requested vote from peers ac9ba2e0575049d7a0bfe53c04902fd7 (127.7.209.67:42523), b8d8aad6ce2d4132ab25f1126999ab29 (127.7.209.66:40341)
I20260929 03:09:34.821839 27106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d6c8060b9a484b9e5d01faaa12ca44" candidate_uuid: "50d064de2baf4a129fa3fb9ec6be2792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7"
I20260929 03:09:34.822007 27106 raft_consensus.cc:3037] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:34.823330 27106 raft_consensus.cc:2454] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d064de2baf4a129fa3fb9ec6be2792 in term 1.
I20260929 03:09:34.823632 26777 leader_election.cc:302] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d064de2baf4a129fa3fb9ec6be2792, ac9ba2e0575049d7a0bfe53c04902fd7; no voters:
I20260929 03:09:34.823592 26975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88d6c8060b9a484b9e5d01faaa12ca44" candidate_uuid: "50d064de2baf4a129fa3fb9ec6be2792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8d8aad6ce2d4132ab25f1126999ab29"
I20260929 03:09:34.823801 27176 raft_consensus.cc:2781] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:34.823741 26975 raft_consensus.cc:3037] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:34.824044 27176 raft_consensus.cc:686] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [term 1 LEADER]: Becoming Leader. State: Replica: 50d064de2baf4a129fa3fb9ec6be2792, State: Running, Role: LEADER
I20260929 03:09:34.824451 27176 consensus_queue.cc:237] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42523 } } peers { permanent_uuid: "b8d8aad6ce2d4132ab25f1126999ab29" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40341 } } peers { permanent_uuid: "50d064de2baf4a129fa3fb9ec6be2792" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44233 } }
I20260929 03:09:34.825184 26975 raft_consensus.cc:2454] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d064de2baf4a129fa3fb9ec6be2792 in term 1.
I20260929 03:09:34.828239 26700 catalog_manager.cc:5495] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50d064de2baf4a129fa3fb9ec6be2792 (127.7.209.65). New cstate: current_term: 1 leader_uuid: "50d064de2baf4a129fa3fb9ec6be2792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8d8aad6ce2d4132ab25f1126999ab29" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50d064de2baf4a129fa3fb9ec6be2792" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 44233 } health_report { overall_health: HEALTHY } } }
I20260929 03:09:34.861527 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:34.858086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39210 (local address 127.7.209.65:44233)
0929 03:09:34.858218 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:34.858222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:34.858253 (+ 31us) server_negotiation.cc:407] Connection header received
0929 03:09:34.858334 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:34.858337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:34.858404 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:34.858491 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:09:34.858788 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.859317 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.860153 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:34.860356 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:34.860425 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:34.860696 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:09:34.861205 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:09:34.861324 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:34.861363 (+ 39us) server_negotiation.cc:299] Negotiation successful
0929 03:09:34.861397 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 03:09:34.868551 26975 raft_consensus.cc:1261] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29 [term 1 FOLLOWER]: Refusing update from remote peer 50d064de2baf4a129fa3fb9ec6be2792: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:09:34.868551 27106 raft_consensus.cc:1261] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7 [term 1 FOLLOWER]: Refusing update from remote peer 50d064de2baf4a129fa3fb9ec6be2792: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:09:34.869408 27180 consensus_queue.cc:1035] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac9ba2e0575049d7a0bfe53c04902fd7" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 42523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:09:34.869637 27176 consensus_queue.cc:1035] T 88d6c8060b9a484b9e5d01faaa12ca44 P 50d064de2baf4a129fa3fb9ec6be2792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8d8aad6ce2d4132ab25f1126999ab29" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 40341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:09:34.888792 26702 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:09:34.889021 26702 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:09:34.891762 26702 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b82b30fa9cb340bfbde99c730edf1ac9: Sending DeleteTablet for 3 replicas of tablet 88d6c8060b9a484b9e5d01faaa12ca44
I20260929 03:09:34.892627 26955 tablet_service.cc:1514] Processing DeleteTablet for tablet 88d6c8060b9a484b9e5d01faaa12ca44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:34 UTC) from {username='slave'} at 127.0.0.1:42554
I20260929 03:09:34.892822 27086 tablet_service.cc:1514] Processing DeleteTablet for tablet 88d6c8060b9a484b9e5d01faaa12ca44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:34 UTC) from {username='slave'} at 127.0.0.1:36234
I20260929 03:09:34.893208 26824 tablet_service.cc:1514] Processing DeleteTablet for tablet 88d6c8060b9a484b9e5d01faaa12ca44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:34 UTC) from {username='slave'} at 127.0.0.1:39196
I20260929 03:09:34.893352 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 26762
I20260929 03:09:34.893597 27192 tablet_replica.cc:330] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29: stopping tablet replica
I20260929 03:09:34.894316 27192 raft_consensus.cc:2229] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:09:34.894816 27192 raft_consensus.cc:2258] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:34.895623 27193 tablet_replica.cc:330] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7: stopping tablet replica
I20260929 03:09:34.895995 27193 raft_consensus.cc:2229] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:09:34.896319 27193 raft_consensus.cc:2258] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:34.897163 27192 ts_tablet_manager.cc:1905] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:09:34.897751 27193 ts_tablet_manager.cc:1905] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:09:34.900352 27193 ts_tablet_manager.cc:1918] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:09:34.900446 27193 log.cc:1196] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/88d6c8060b9a484b9e5d01faaa12ca44
I20260929 03:09:34.900529 27192 ts_tablet_manager.cc:1918] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:09:34.900635 27192 log.cc:1196] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/88d6c8060b9a484b9e5d01faaa12ca44
I20260929 03:09:34.900774 27193 ts_tablet_manager.cc:1939] T 88d6c8060b9a484b9e5d01faaa12ca44 P ac9ba2e0575049d7a0bfe53c04902fd7: Deleting consensus metadata
I20260929 03:09:34.900947 27192 ts_tablet_manager.cc:1939] T 88d6c8060b9a484b9e5d01faaa12ca44 P b8d8aad6ce2d4132ab25f1126999ab29: Deleting consensus metadata
W20260929 03:09:34.902138 26686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:44233 (error 108)
I20260929 03:09:34.902175 26688 catalog_manager.cc:4841] TS b8d8aad6ce2d4132ab25f1126999ab29 (127.7.209.66:40341): tablet 88d6c8060b9a484b9e5d01faaa12ca44 (table test-table [id=6ba5ecf12b804dbfa3bd9873881aa047]) successfully deleted
I20260929 03:09:34.902688 26686 catalog_manager.cc:4841] TS ac9ba2e0575049d7a0bfe53c04902fd7 (127.7.209.67:42523): tablet 88d6c8060b9a484b9e5d01faaa12ca44 (table test-table [id=6ba5ecf12b804dbfa3bd9873881aa047]) successfully deleted
W20260929 03:09:34.903340 27164 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:09:34.902993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:44233 (local address 127.0.0.1:39212)
0929 03:09:34.903135 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:34.903205 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:44233: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 03:09:34.903375 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 26893
W20260929 03:09:34.903532 26686 catalog_manager.cc:4584] TS 50d064de2baf4a129fa3fb9ec6be2792 (127.7.209.65:44233): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88d6c8060b9a484b9e5d01faaa12ca44: Network error: Client connection negotiation failed: client connection to 127.7.209.65:44233: connect: Connection reset by peer (error 104)
I20260929 03:09:34.910125 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 27024
I20260929 03:09:34.916026 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 26671
2026-09-29T03:09:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1187 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:34 dist-test-slave-bdps krb5kdc[27197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:34 dist-test-slave-bdps krb5kdc[27197](info): set up 2 sockets
Sep 29 03:09:34 dist-test-slave-bdps krb5kdc[27197](info): commencing operation
krb5kdc: starting...
W20260929 03:09:35.874002 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.936s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:35 dist-test-slave-bdps krb5kdc[27197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:09:36.002485 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:40801
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:40725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:40801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:36.127146 27210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:36.127451 27210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:36.127493 27210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:36.131590 27210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:36.131657 27210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:36.131676 27210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:36.131691 27210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:36.131705 27210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:36.136890 27210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:40725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:40801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:40801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:36.138084 27210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:36.139312 27210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:36.148677 27216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:36.148928 27210 server_base.cc:1034] running on GCE node
W20260929 03:09:36.149087 27215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:36.149094 27218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:36.149933 27210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:36.151522 27210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:36.152695 27210 hybrid_clock.cc:648] HybridClock initialized: now 1790651376152670 us; error 44 us; skew 500 ppm
Sep 29 03:09:36 dist-test-slave-bdps krb5kdc[27197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651376, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:36.155910 27210 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:09:36.156997 27210 webserver.cc:460] Webserver started at http://127.7.209.126:35993/ using document root <none> and password file <none>
I20260929 03:09:36.157684 27210 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:36.157756 27210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:36.157972 27210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:36.160461 27210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "efea5aaa7a0641dbbab63c4a662e79cf"
format_stamp: "Formatted at 2026-09-29 03:09:36 on dist-test-slave-bdps"
server_key: "f76ae6f5cb17809a987e5267d94d3e25"
server_key_iv: "7a751a8a3f31f05f8f01035470b3d0d4"
server_key_version: "encryptionkey@0"
I20260929 03:09:36.161074 27210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "efea5aaa7a0641dbbab63c4a662e79cf"
format_stamp: "Formatted at 2026-09-29 03:09:36 on dist-test-slave-bdps"
server_key: "f76ae6f5cb17809a987e5267d94d3e25"
server_key_iv: "7a751a8a3f31f05f8f01035470b3d0d4"
server_key_version: "encryptionkey@0"
I20260929 03:09:36.165385 27210 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260929 03:09:36.167987 27225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:36.169427 27210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:09:36.169559 27210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "efea5aaa7a0641dbbab63c4a662e79cf"
format_stamp: "Formatted at 2026-09-29 03:09:36 on dist-test-slave-bdps"
server_key: "f76ae6f5cb17809a987e5267d94d3e25"
server_key_iv: "7a751a8a3f31f05f8f01035470b3d0d4"
server_key_version: "encryptionkey@0"
I20260929 03:09:36.169721 27210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:36.182670 27210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:36.186076 27210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:36.186272 27210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:36.194031 27210 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:40801
I20260929 03:09:36.194072 27277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:40801 every 8 connection(s)
I20260929 03:09:36.195240 27210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:36.198722 27278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:36.199851 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 27210
I20260929 03:09:36.200006 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:36.200412 8005 external_mini_cluster.cc:1428] Setting key dd40ccdfe13daab0b254784df367140f
I20260929 03:09:36.205914 27278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf: Bootstrap starting.
I20260929 03:09:36.208586 27278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:36.209388 27278 log.cc:824] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf: Log is configured to *not* fsync() on all Append() calls
Sep 29 03:09:36 dist-test-slave-bdps krb5kdc[27197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:09:36.211659 27278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf: No bootstrap required, opened a new log
I20260929 03:09:36.215487 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:36.202188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42852 (local address 127.7.209.126:40801)
0929 03:09:36.202662 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:36.202679 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:36.202721 (+ 42us) server_negotiation.cc:407] Connection header received
0929 03:09:36.203420 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:36.203440 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:36.203833 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:36.204200 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:36.205114 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:36.206277 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:36.207200 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:36.207733 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:36.207894 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:36.210703 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:36.210708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:36.210724 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:36.210759 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:36.212521 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:36.212943 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:36.212949 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:36.212955 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:36.213073 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:36.213350 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:36.213353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:36.213355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:36.213842 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:36.214043 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:36.214311 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 03:09:36.214636 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":139,"threads_started":1}
I20260929 03:09:36.215549 27278 raft_consensus.cc:348] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efea5aaa7a0641dbbab63c4a662e79cf" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40801 } }
I20260929 03:09:36.215880 27278 raft_consensus.cc:374] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:36.215940 27278 raft_consensus.cc:729] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efea5aaa7a0641dbbab63c4a662e79cf, State: Initialized, Role: FOLLOWER
I20260929 03:09:36.216444 27278 consensus_queue.cc:260] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efea5aaa7a0641dbbab63c4a662e79cf" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40801 } }
I20260929 03:09:36.216606 27278 raft_consensus.cc:388] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:36.216670 27278 raft_consensus.cc:482] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:36.216779 27278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:36.217737 27278 raft_consensus.cc:504] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efea5aaa7a0641dbbab63c4a662e79cf" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40801 } }
I20260929 03:09:36.218120 27278 leader_election.cc:302] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efea5aaa7a0641dbbab63c4a662e79cf; no voters:
I20260929 03:09:36.218472 27278 leader_election.cc:288] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:36.218578 27283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:36.218786 27283 raft_consensus.cc:686] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [term 1 LEADER]: Becoming Leader. State: Replica: efea5aaa7a0641dbbab63c4a662e79cf, State: Running, Role: LEADER
I20260929 03:09:36.219121 27283 consensus_queue.cc:237] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efea5aaa7a0641dbbab63c4a662e79cf" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40801 } }
I20260929 03:09:36.219780 27278 sys_catalog.cc:564] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:36.220853 27285 sys_catalog.cc:455] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader efea5aaa7a0641dbbab63c4a662e79cf. Latest consensus state: current_term: 1 leader_uuid: "efea5aaa7a0641dbbab63c4a662e79cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efea5aaa7a0641dbbab63c4a662e79cf" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40801 } } }
I20260929 03:09:36.220990 27285 sys_catalog.cc:458] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:36.221300 27284 sys_catalog.cc:455] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efea5aaa7a0641dbbab63c4a662e79cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efea5aaa7a0641dbbab63c4a662e79cf" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40801 } } }
I20260929 03:09:36.221395 27284 sys_catalog.cc:458] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:36.221510 27292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:36.225009 27292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:36.231626 27292 catalog_manager.cc:1347] Generated new cluster ID: eea80c6f070e472094c9a1e44f170ca1
I20260929 03:09:36.231700 27292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:36.256021 27292 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:36.257157 27292 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:36.266409 27292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf: Generated new TSK 0
I20260929 03:09:36.267300 27292 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.209.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
Entry for principal kudu/127.7.209.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab.
I20260929 03:09:36.308462 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:40801
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:40725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:36.431789 27304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:36.432170 27304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:36.432238 27304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:36.436507 27304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:36.436582 27304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:36.436668 27304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:09:36.442281 27304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:40725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.65.keytab
--principal=kudu/127.7.209.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:40801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:36.443534 27304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:36.444754 27304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:36.455082 27310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:36.455081 27309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:36.455106 27312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:36.455192 27304 server_base.cc:1034] running on GCE node
I20260929 03:09:36.455753 27304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:36.456424 27304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:36.457583 27304 hybrid_clock.cc:648] HybridClock initialized: now 1790651376457567 us; error 30 us; skew 500 ppm
Sep 29 03:09:36 dist-test-slave-bdps krb5kdc[27197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651376, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:36.460696 27304 init.cc:377] Logged in from keytab as kudu/127.7.209.65@KRBTEST.COM (short username kudu)
I20260929 03:09:36.461678 27304 webserver.cc:460] Webserver started at http://127.7.209.65:35367/ using document root <none> and password file <none>
I20260929 03:09:36.462311 27304 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:36.462379 27304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:36.462608 27304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:36.464999 27304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "61623a6e3f344baea8a5a811a4fd68f9"
format_stamp: "Formatted at 2026-09-29 03:09:36 on dist-test-slave-bdps"
server_key: "7072844b4988b0dbd25fb63d17ae0faa"
server_key_iv: "109a7a01b0763bc0a5b4a4e8a0690fbd"
server_key_version: "encryptionkey@0"
I20260929 03:09:36.465600 27304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "61623a6e3f344baea8a5a811a4fd68f9"
format_stamp: "Formatted at 2026-09-29 03:09:36 on dist-test-slave-bdps"
server_key: "7072844b4988b0dbd25fb63d17ae0faa"
server_key_iv: "109a7a01b0763bc0a5b4a4e8a0690fbd"
server_key_version: "encryptionkey@0"
I20260929 03:09:36.470216 27304 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:09:36.473009 27319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:36.474547 27304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:36.474660 27304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "61623a6e3f344baea8a5a811a4fd68f9"
format_stamp: "Formatted at 2026-09-29 03:09:36 on dist-test-slave-bdps"
server_key: "7072844b4988b0dbd25fb63d17ae0faa"
server_key_iv: "109a7a01b0763bc0a5b4a4e8a0690fbd"
server_key_version: "encryptionkey@0"
I20260929 03:09:36.474773 27304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:36.501958 27304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:36.505714 27304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:36.505925 27304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:36.506610 27304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:36.507555 27304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:36.507602 27304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:36.507650 27304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:36.507690 27304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:36.517802 27304 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:33157
I20260929 03:09:36.517843 27432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:33157 every 8 connection(s)
I20260929 03:09:36.518882 27304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:09:36.525828 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 27304
I20260929 03:09:36.525957 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:09:36.526371 8005 external_mini_cluster.cc:1428] Setting key 5a58ae6163a29af1f8759c173d842580
Sep 29 03:09:36 dist-test-slave-bdps krb5kdc[27197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651376, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.65@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:09:36.534926 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:36.521025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:43459 (local address 127.7.209.126:40801)
0929 03:09:36.521179 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:36.521183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:36.522275 (+ 1092us) server_negotiation.cc:407] Connection header received
0929 03:09:36.523734 (+ 1459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:36.523738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:36.523802 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:36.523888 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:36.525288 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:36.525940 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:36.527494 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:36.527826 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:36.527937 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:36.531075 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:36.531080 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:36.531082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:36.531114 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:36.532690 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:36.533192 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:36.533195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:36.533198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:36.533297 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:36.533646 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:36.533649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:36.533651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:36.533908 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:36.534018 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:36.534566 (+ 548us) server_negotiation.cc:299] Negotiation successful
0929 03:09:36.534728 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 03:09:36.535832 27435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:36.521550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:40801 (local address 127.7.209.65:43459)
0929 03:09:36.522118 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:36.522168 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:36.523500 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:36.524101 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:36.524110 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:36.524527 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:36.525117 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:36.525134 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:36.526090 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:36.526095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:36.527353 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:36.527365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:36.527985 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:36.527989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:36.528220 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:36.528952 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:09:36.528974 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:36.530888 (+ 1914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:36.532805 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:36.532811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:36.532825 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:36.533089 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:36.533380 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:36.533382 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:36.533383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:36.533516 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:36.534014 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:36.534020 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:36.534311 (+ 291us) client_negotiation.cc:769] Sending connection context
0929 03:09:36.534490 (+ 179us) client_negotiation.cc:240] Negotiation successful
0929 03:09:36.534793 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":145,"threads_started":1}
I20260929 03:09:36.537225 27433 heartbeater.cc:347] Connected to a master server at 127.7.209.126:40801
I20260929 03:09:36.537528 27433 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.209.66@KRBTEST.COM; defaulting to no policy
I20260929 03:09:36.538250 27433 heartbeater.cc:511] Master 127.7.209.126:40801 requested a full tablet report, sending...
I20260929 03:09:36.540082 27242 ts_manager.cc:194] Registered new tserver with Master: 61623a6e3f344baea8a5a811a4fd68f9 (127.7.209.65:33157)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.66@KRBTEST.COM" created.
I20260929 03:09:36.541491 27242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.65@KRBTEST.COM'} at 127.7.209.65:43459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
Entry for principal kudu/127.7.209.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab.
I20260929 03:09:36.555001 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:40801
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:40725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:36.680083 27438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:36.680436 27438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:36.680490 27438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:36.684882 27438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:36.684959 27438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:36.685024 27438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:09:36.691092 27438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:40725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.66.keytab
--principal=kudu/127.7.209.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:40801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:36.692570 27438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:36.693910 27438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:36.705068 27446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:36.705057 27443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:36.705176 27438 server_base.cc:1034] running on GCE node
W20260929 03:09:36.705096 27444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:36.705760 27438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:36.706453 27438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:36.707610 27438 hybrid_clock.cc:648] HybridClock initialized: now 1790651376707589 us; error 37 us; skew 500 ppm
Sep 29 03:09:36 dist-test-slave-bdps krb5kdc[27197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651376, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:36.710569 27438 init.cc:377] Logged in from keytab as kudu/127.7.209.66@KRBTEST.COM (short username kudu)
I20260929 03:09:36.711583 27438 webserver.cc:460] Webserver started at http://127.7.209.66:46349/ using document root <none> and password file <none>
I20260929 03:09:36.712273 27438 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:36.712338 27438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:36.712582 27438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:36.715081 27438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe"
format_stamp: "Formatted at 2026-09-29 03:09:36 on dist-test-slave-bdps"
server_key: "c0fc7f50b535462d1dca6110111a6575"
server_key_iv: "cec1d98c2c15ea7c2c45850e50491b97"
server_key_version: "encryptionkey@0"
I20260929 03:09:36.715703 27438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe"
format_stamp: "Formatted at 2026-09-29 03:09:36 on dist-test-slave-bdps"
server_key: "c0fc7f50b535462d1dca6110111a6575"
server_key_iv: "cec1d98c2c15ea7c2c45850e50491b97"
server_key_version: "encryptionkey@0"
I20260929 03:09:36.720157 27438 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:09:36.723054 27453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:36.724386 27438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:09:36.724483 27438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe"
format_stamp: "Formatted at 2026-09-29 03:09:36 on dist-test-slave-bdps"
server_key: "c0fc7f50b535462d1dca6110111a6575"
server_key_iv: "cec1d98c2c15ea7c2c45850e50491b97"
server_key_version: "encryptionkey@0"
I20260929 03:09:36.724567 27438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:36.768446 27438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:36.771808 27438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:36.771989 27438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:36.772656 27438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:36.773715 27438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:36.773772 27438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:36.773825 27438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:36.773845 27438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:36.783859 27438 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:38627
I20260929 03:09:36.783900 27566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:38627 every 8 connection(s)
I20260929 03:09:36.784876 27438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:09:36.793639 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 27438
I20260929 03:09:36.793795 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:09:36.794124 8005 external_mini_cluster.cc:1428] Setting key ead6557a9f1f6c0737e04b3a3b304f5f
Sep 29 03:09:36 dist-test-slave-bdps krb5kdc[27197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651376, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.66@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:09:36.800207 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:36.786903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:57959 (local address 127.7.209.126:40801)
0929 03:09:36.787051 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:36.787055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:36.788018 (+ 963us) server_negotiation.cc:407] Connection header received
0929 03:09:36.789500 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:36.789504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:36.789583 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:36.789699 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:36.791154 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:36.791751 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:36.793042 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:36.793317 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:36.793429 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:36.796333 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:36.796336 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:36.796339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:36.796367 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:36.797846 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:36.798474 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:36.798477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:36.798479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:36.798571 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:36.798941 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:36.798943 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:36.798945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:36.799134 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:36.799208 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:36.799888 (+ 680us) server_negotiation.cc:299] Negotiation successful
0929 03:09:36.800024 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 03:09:36.801215 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:36.787361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:40801 (local address 127.7.209.66:57959)
0929 03:09:36.787861 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:36.787909 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:36.789255 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:36.789927 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:36.789936 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:36.790375 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:36.790994 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:36.791010 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:36.791888 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:36.791891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:36.792914 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:36.792924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:36.793513 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:36.793516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:36.793725 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:36.794386 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:09:36.794404 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:36.796139 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:36.797954 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:36.797961 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:36.797982 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:36.798357 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:36.798656 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:36.798659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:36.798661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:36.798859 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:36.799220 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:36.799228 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:36.799600 (+ 372us) client_negotiation.cc:769] Sending connection context
0929 03:09:36.799839 (+ 239us) client_negotiation.cc:240] Negotiation successful
0929 03:09:36.800163 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":134,"threads_started":1}
I20260929 03:09:36.802560 27567 heartbeater.cc:347] Connected to a master server at 127.7.209.126:40801
I20260929 03:09:36.802863 27567 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:36.803499 27567 heartbeater.cc:511] Master 127.7.209.126:40801 requested a full tablet report, sending...
I20260929 03:09:36.804760 27242 ts_manager.cc:194] Registered new tserver with Master: 0e0cfd4ca56c4e87b02c011c52d8f4fe (127.7.209.66:38627)
WARNING: no policy specified for kudu/127.7.209.67@KRBTEST.COM; defaulting to no policy
I20260929 03:09:36.805691 27242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.66@KRBTEST.COM'} at 127.7.209.66:57959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
Entry for principal kudu/127.7.209.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab.
I20260929 03:09:36.822091 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:40801
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:40725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:36.944049 27572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:36.944306 27572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:36.944346 27572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:36.948390 27572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:36.948446 27572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:36.948499 27572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:09:36.953727 27572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:40725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.67.keytab
--principal=kudu/127.7.209.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:40801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:36.954957 27572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:36.956156 27572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:09:36.967106 27572 server_base.cc:1034] running on GCE node
W20260929 03:09:36.967095 27578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:36.967170 27577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:36.967099 27580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:36.967851 27572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:36.968547 27572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:36.969662 27572 hybrid_clock.cc:648] HybridClock initialized: now 1790651376969637 us; error 45 us; skew 500 ppm
Sep 29 03:09:36 dist-test-slave-bdps krb5kdc[27197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651376, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:36.973001 27572 init.cc:377] Logged in from keytab as kudu/127.7.209.67@KRBTEST.COM (short username kudu)
I20260929 03:09:36.974052 27572 webserver.cc:460] Webserver started at http://127.7.209.67:35567/ using document root <none> and password file <none>
I20260929 03:09:36.974680 27572 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:36.974757 27572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:36.974987 27572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:36.977562 27572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "6d8d873a93f2422489d8599536b01df2"
format_stamp: "Formatted at 2026-09-29 03:09:36 on dist-test-slave-bdps"
server_key: "4373752da4f93cd47faff99d2a25601a"
server_key_iv: "2f3ec1d6bb687e8a9e7b2eaa2df7f437"
server_key_version: "encryptionkey@0"
I20260929 03:09:36.978309 27572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "6d8d873a93f2422489d8599536b01df2"
format_stamp: "Formatted at 2026-09-29 03:09:36 on dist-test-slave-bdps"
server_key: "4373752da4f93cd47faff99d2a25601a"
server_key_iv: "2f3ec1d6bb687e8a9e7b2eaa2df7f437"
server_key_version: "encryptionkey@0"
I20260929 03:09:36.982921 27572 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:09:36.985910 27587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:36.987468 27572 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 03:09:36.987603 27572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "6d8d873a93f2422489d8599536b01df2"
format_stamp: "Formatted at 2026-09-29 03:09:36 on dist-test-slave-bdps"
server_key: "4373752da4f93cd47faff99d2a25601a"
server_key_iv: "2f3ec1d6bb687e8a9e7b2eaa2df7f437"
server_key_version: "encryptionkey@0"
I20260929 03:09:36.987751 27572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:37.012792 27572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:37.016274 27572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:37.016466 27572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:37.017189 27572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:37.018280 27572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:37.018349 27572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:37.018417 27572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:37.018446 27572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:37.028213 27572 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:35999
I20260929 03:09:37.028256 27700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:35999 every 8 connection(s)
I20260929 03:09:37.029341 27572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:09:37.038968 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 27572
I20260929 03:09:37.039131 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:09:37.039538 8005 external_mini_cluster.cc:1428] Setting key 69595f078ed316fe5585d3b7000f4a30
Sep 29 03:09:37 dist-test-slave-bdps krb5kdc[27197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651376, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.67@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:09:37.044046 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.031425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:47067 (local address 127.7.209.126:40801)
0929 03:09:37.031574 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:37.031578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:37.032576 (+ 998us) server_negotiation.cc:407] Connection header received
0929 03:09:37.034124 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:37.034129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:37.034200 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:37.034290 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:37.035693 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.036282 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.037584 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.037864 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.037964 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.040832 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:37.040835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:37.040838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:37.040863 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:37.042348 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:37.042793 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:37.042795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:37.042796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:37.042865 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:37.043134 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:37.043136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:37.043137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:37.043303 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:37.043373 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:37.043814 (+ 441us) server_negotiation.cc:299] Negotiation successful
0929 03:09:37.043914 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:09:37.044971 27703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.031891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:40801 (local address 127.7.209.67:47067)
0929 03:09:37.032395 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:37.032446 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:37.033862 (+ 1416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:37.034508 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:37.034516 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:37.034946 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:37.035533 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.035549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.036419 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.036423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.037460 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.037470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.038015 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.038018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.038192 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.038806 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:09:37.038827 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:37.040675 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:37.042444 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:37.042448 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:37.042461 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:37.042711 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:37.042940 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:09:37.042942 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:09:37.042943 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:09:37.043070 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:09:37.043371 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:37.043376 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:37.043612 (+ 236us) client_negotiation.cc:769] Sending connection context
0929 03:09:37.043784 (+ 172us) client_negotiation.cc:240] Negotiation successful
0929 03:09:37.044050 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":134,"threads_started":1}
I20260929 03:09:37.046072 27701 heartbeater.cc:347] Connected to a master server at 127.7.209.126:40801
I20260929 03:09:37.046317 27701 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:37.046818 27701 heartbeater.cc:511] Master 127.7.209.126:40801 requested a full tablet report, sending...
I20260929 03:09:37.047856 27242 ts_manager.cc:194] Registered new tserver with Master: 6d8d873a93f2422489d8599536b01df2 (127.7.209.67:35999)
I20260929 03:09:37.048432 27242 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.209.67@KRBTEST.COM'} at 127.7.209.67:47067
I20260929 03:09:37.053273 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:09:37.062184 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.055452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42856 (local address 127.7.209.126:40801)
0929 03:09:37.055557 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:37.055561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:37.055732 (+ 171us) server_negotiation.cc:407] Connection header received
0929 03:09:37.055849 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:37.055852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:37.055895 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:37.055960 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:37.056656 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.057166 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.058052 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.058279 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.058363 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.059549 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:37.059553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:37.059556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:37.059580 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:37.060681 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:37.061060 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:37.061064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:37.061066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:37.061136 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:37.061372 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:37.061375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:37.061376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:37.061586 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:37.061749 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:37.061935 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 03:09:37.062037 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 03:09:37.064678 27242 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:09:37.068285 27242 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:09:37.084321 27715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.078051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:38627 (local address 127.0.0.1:38456)
0929 03:09:37.079195 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:37.079264 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:37.079410 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:37.079978 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:37.079985 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:37.080010 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:37.080173 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.080181 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.081438 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.081442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.082672 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.082681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.083727 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.083731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.083850 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.083893 (+ 43us) client_negotiation.cc:769] Sending connection context
0929 03:09:37.084018 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 03:09:37.084150 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"thread_start_us":103,"threads_started":1}
I20260929 03:09:37.084354 27716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.078912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38456 (local address 127.7.209.66:38627)
0929 03:09:37.079477 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:37.079492 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:37.079521 (+ 29us) server_negotiation.cc:407] Connection header received
0929 03:09:37.079599 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:37.079603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:37.079813 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:37.079984 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:37.080315 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.081321 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.082798 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.083601 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.083718 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.083897 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:37.084061 (+ 164us) server_negotiation.cc:299] Negotiation successful
0929 03:09:37.084175 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":144,"threads_started":1}
I20260929 03:09:37.084827 27712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.078448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:35999 (local address 127.0.0.1:42308)
0929 03:09:37.080020 (+ 1572us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:37.080033 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:37.080124 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:37.080637 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:37.080640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:37.080658 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:37.080762 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.080767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.082074 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.082077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.083189 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.083196 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.084237 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.084251 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.084547 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.084577 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 03:09:37.084638 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 03:09:37.084710 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1520,"mutex_wait_us":25,"thread_start_us":101,"threads_started":1}
I20260929 03:09:37.085139 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.078431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56706 (local address 127.7.209.65:33157)
0929 03:09:37.079711 (+ 1280us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:37.079719 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:37.079768 (+ 49us) server_negotiation.cc:407] Connection header received
0929 03:09:37.079853 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:37.079859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:37.080069 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:37.080262 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:37.080457 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.081646 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.083134 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.083951 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.084048 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.084180 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:37.084872 (+ 692us) server_negotiation.cc:299] Negotiation successful
0929 03:09:37.084993 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1171,"thread_start_us":83,"threads_started":1}
I20260929 03:09:37.085561 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.078990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42308 (local address 127.7.209.67:35999)
0929 03:09:37.079902 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:37.079914 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:37.080214 (+ 300us) server_negotiation.cc:407] Connection header received
0929 03:09:37.080301 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:37.080306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:37.080495 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:37.080648 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:37.080840 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.081932 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.083292 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.083948 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.084038 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.084178 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:37.085118 (+ 940us) server_negotiation.cc:299] Negotiation successful
0929 03:09:37.085406 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":807,"thread_start_us":166,"threads_started":1}
I20260929 03:09:37.085023 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.078145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:33157 (local address 127.0.0.1:56706)
0929 03:09:37.079524 (+ 1379us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:37.079537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:37.079618 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:37.080238 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:37.080242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:37.080259 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:37.080365 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.080372 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.081777 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.081780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.083010 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.083019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.084243 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.084247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.084397 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.084422 (+ 25us) client_negotiation.cc:769] Sending connection context
0929 03:09:37.084820 (+ 398us) client_negotiation.cc:240] Negotiation successful
0929 03:09:37.084898 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1314,"mutex_wait_us":34,"thread_start_us":145,"threads_started":1}
I20260929 03:09:37.088153 27501 tablet_service.cc:1467] Processing CreateTablet for tablet c6b4e5cbd3b54f63bfb338b11ec0f750 (DEFAULT_TABLE table=test-table [id=d0dae62a2fa84452be6f8bf648989291]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:37.089031 27367 tablet_service.cc:1467] Processing CreateTablet for tablet c6b4e5cbd3b54f63bfb338b11ec0f750 (DEFAULT_TABLE table=test-table [id=d0dae62a2fa84452be6f8bf648989291]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:37.089569 27501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6b4e5cbd3b54f63bfb338b11ec0f750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:37.090276 27367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6b4e5cbd3b54f63bfb338b11ec0f750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:37.090723 27635 tablet_service.cc:1467] Processing CreateTablet for tablet c6b4e5cbd3b54f63bfb338b11ec0f750 (DEFAULT_TABLE table=test-table [id=d0dae62a2fa84452be6f8bf648989291]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:37.091642 27635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6b4e5cbd3b54f63bfb338b11ec0f750. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:37.098053 27718 tablet_bootstrap.cc:492] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2: Bootstrap starting.
I20260929 03:09:37.099325 27719 tablet_bootstrap.cc:492] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe: Bootstrap starting.
I20260929 03:09:37.100450 27718 tablet_bootstrap.cc:654] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:37.101661 27718 log.cc:824] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:37.101728 27719 tablet_bootstrap.cc:654] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:37.101951 27720 tablet_bootstrap.cc:492] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9: Bootstrap starting.
I20260929 03:09:37.102654 27719 log.cc:824] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:37.104007 27718 tablet_bootstrap.cc:492] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2: No bootstrap required, opened a new log
I20260929 03:09:37.104229 27718 ts_tablet_manager.cc:1397] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260929 03:09:37.104261 27720 tablet_bootstrap.cc:654] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:37.104933 27719 tablet_bootstrap.cc:492] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe: No bootstrap required, opened a new log
I20260929 03:09:37.104943 27720 log.cc:824] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:37.105134 27719 ts_tablet_manager.cc:1397] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260929 03:09:37.106769 27720 tablet_bootstrap.cc:492] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9: No bootstrap required, opened a new log
I20260929 03:09:37.106945 27720 ts_tablet_manager.cc:1397] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260929 03:09:37.108753 27718 raft_consensus.cc:348] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38627 } } peers { permanent_uuid: "61623a6e3f344baea8a5a811a4fd68f9" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33157 } } peers { permanent_uuid: "6d8d873a93f2422489d8599536b01df2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35999 } }
I20260929 03:09:37.109097 27718 raft_consensus.cc:374] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:37.109160 27718 raft_consensus.cc:729] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d8d873a93f2422489d8599536b01df2, State: Initialized, Role: FOLLOWER
I20260929 03:09:37.109526 27719 raft_consensus.cc:348] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38627 } } peers { permanent_uuid: "61623a6e3f344baea8a5a811a4fd68f9" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33157 } } peers { permanent_uuid: "6d8d873a93f2422489d8599536b01df2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35999 } }
I20260929 03:09:37.109838 27719 raft_consensus.cc:374] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:37.109895 27719 raft_consensus.cc:729] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0cfd4ca56c4e87b02c011c52d8f4fe, State: Initialized, Role: FOLLOWER
I20260929 03:09:37.109855 27718 consensus_queue.cc:260] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38627 } } peers { permanent_uuid: "61623a6e3f344baea8a5a811a4fd68f9" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33157 } } peers { permanent_uuid: "6d8d873a93f2422489d8599536b01df2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35999 } }
I20260929 03:09:37.110428 27719 consensus_queue.cc:260] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38627 } } peers { permanent_uuid: "61623a6e3f344baea8a5a811a4fd68f9" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33157 } } peers { permanent_uuid: "6d8d873a93f2422489d8599536b01df2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35999 } }
I20260929 03:09:37.110845 27720 raft_consensus.cc:348] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38627 } } peers { permanent_uuid: "61623a6e3f344baea8a5a811a4fd68f9" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33157 } } peers { permanent_uuid: "6d8d873a93f2422489d8599536b01df2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35999 } }
I20260929 03:09:37.111042 27718 ts_tablet_manager.cc:1428] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260929 03:09:37.111096 27720 raft_consensus.cc:374] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:37.111153 27720 raft_consensus.cc:729] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61623a6e3f344baea8a5a811a4fd68f9, State: Initialized, Role: FOLLOWER
I20260929 03:09:37.111449 27701 heartbeater.cc:503] Master 127.7.209.126:40801 was elected leader, sending a full tablet report...
I20260929 03:09:37.111470 27719 ts_tablet_manager.cc:1428] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 03:09:37.111610 27720 consensus_queue.cc:260] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38627 } } peers { permanent_uuid: "61623a6e3f344baea8a5a811a4fd68f9" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33157 } } peers { permanent_uuid: "6d8d873a93f2422489d8599536b01df2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35999 } }
I20260929 03:09:37.111830 27567 heartbeater.cc:503] Master 127.7.209.126:40801 was elected leader, sending a full tablet report...
I20260929 03:09:37.112460 27433 heartbeater.cc:503] Master 127.7.209.126:40801 was elected leader, sending a full tablet report...
I20260929 03:09:37.113361 27720 ts_tablet_manager.cc:1428] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260929 03:09:37.157924 27725 raft_consensus.cc:482] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:09:37.158254 27725 raft_consensus.cc:504] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38627 } } peers { permanent_uuid: "61623a6e3f344baea8a5a811a4fd68f9" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33157 } } peers { permanent_uuid: "6d8d873a93f2422489d8599536b01df2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35999 } }
I20260929 03:09:37.159790 27725 leader_election.cc:288] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61623a6e3f344baea8a5a811a4fd68f9 (127.7.209.65:33157), 6d8d873a93f2422489d8599536b01df2 (127.7.209.67:35999)
I20260929 03:09:37.165194 27714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.160306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:34193 (local address 127.7.209.65:33157)
0929 03:09:37.160466 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:37.160473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:37.160647 (+ 174us) server_negotiation.cc:407] Connection header received
0929 03:09:37.160746 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:37.160749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:37.160801 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:37.160875 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:37.161120 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.162006 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.163881 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.164570 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.164672 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.164732 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:37.164949 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 03:09:37.165035 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 03:09:37.165194 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.160201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:36271 (local address 127.7.209.67:35999)
0929 03:09:37.160572 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:37.160576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:37.160588 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:09:37.160648 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:37.160651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:37.160705 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:37.160803 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:37.161136 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.162006 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.163875 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.164529 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.164667 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.164732 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:37.164949 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 03:09:37.165033 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271}
I20260929 03:09:37.165210 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.160040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:35999 (local address 127.7.209.66:36271)
0929 03:09:37.160188 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:37.160205 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:37.160312 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:37.160819 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:37.160821 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:37.160838 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:37.161017 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.161025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.162124 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.162127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.163737 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.163757 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.164652 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.164656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.164798 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.164852 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 03:09:37.164929 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 03:09:37.165031 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":215}
I20260929 03:09:37.165220 27727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.160041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:33157 (local address 127.7.209.66:34193)
0929 03:09:37.160560 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:37.160571 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:37.160667 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:37.160889 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:37.160891 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:37.160903 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:37.161025 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.161030 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.162124 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.162127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.163742 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.163750 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.164668 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.164671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.164781 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.164849 (+ 68us) client_negotiation.cc:769] Sending connection context
0929 03:09:37.164930 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 03:09:37.165031 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":280,"thread_start_us":163,"threads_started":1}
I20260929 03:09:37.165961 27655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b4e5cbd3b54f63bfb338b11ec0f750" candidate_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d8d873a93f2422489d8599536b01df2" is_pre_election: true
I20260929 03:09:37.165963 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b4e5cbd3b54f63bfb338b11ec0f750" candidate_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61623a6e3f344baea8a5a811a4fd68f9" is_pre_election: true
I20260929 03:09:37.166301 27655 raft_consensus.cc:2454] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e0cfd4ca56c4e87b02c011c52d8f4fe in term 0.
I20260929 03:09:37.166301 27387 raft_consensus.cc:2454] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e0cfd4ca56c4e87b02c011c52d8f4fe in term 0.
I20260929 03:09:37.166792 27454 leader_election.cc:302] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e0cfd4ca56c4e87b02c011c52d8f4fe, 61623a6e3f344baea8a5a811a4fd68f9; no voters:
I20260929 03:09:37.167086 27725 raft_consensus.cc:2781] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:09:37.167182 27725 raft_consensus.cc:482] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:09:37.167232 27725 raft_consensus.cc:3037] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:37.168522 27725 raft_consensus.cc:504] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38627 } } peers { permanent_uuid: "61623a6e3f344baea8a5a811a4fd68f9" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33157 } } peers { permanent_uuid: "6d8d873a93f2422489d8599536b01df2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35999 } }
I20260929 03:09:37.168898 27725 leader_election.cc:288] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [CANDIDATE]: Term 1 election: Requested vote from peers 61623a6e3f344baea8a5a811a4fd68f9 (127.7.209.65:33157), 6d8d873a93f2422489d8599536b01df2 (127.7.209.67:35999)
I20260929 03:09:37.169272 27655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b4e5cbd3b54f63bfb338b11ec0f750" candidate_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d8d873a93f2422489d8599536b01df2"
I20260929 03:09:37.169305 27387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6b4e5cbd3b54f63bfb338b11ec0f750" candidate_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61623a6e3f344baea8a5a811a4fd68f9"
I20260929 03:09:37.169410 27655 raft_consensus.cc:3037] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:37.169440 27387 raft_consensus.cc:3037] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:37.170778 27387 raft_consensus.cc:2454] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e0cfd4ca56c4e87b02c011c52d8f4fe in term 1.
I20260929 03:09:37.170796 27655 raft_consensus.cc:2454] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e0cfd4ca56c4e87b02c011c52d8f4fe in term 1.
I20260929 03:09:37.171098 27454 leader_election.cc:302] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0e0cfd4ca56c4e87b02c011c52d8f4fe, 61623a6e3f344baea8a5a811a4fd68f9, 6d8d873a93f2422489d8599536b01df2; no voters:
I20260929 03:09:37.171293 27725 raft_consensus.cc:2781] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:37.171581 27725 raft_consensus.cc:686] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 1 LEADER]: Becoming Leader. State: Replica: 0e0cfd4ca56c4e87b02c011c52d8f4fe, State: Running, Role: LEADER
I20260929 03:09:37.171959 27725 consensus_queue.cc:237] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38627 } } peers { permanent_uuid: "61623a6e3f344baea8a5a811a4fd68f9" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33157 } } peers { permanent_uuid: "6d8d873a93f2422489d8599536b01df2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35999 } }
I20260929 03:09:37.175813 27241 catalog_manager.cc:5495] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e0cfd4ca56c4e87b02c011c52d8f4fe (127.7.209.66). New cstate: current_term: 1 leader_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0cfd4ca56c4e87b02c011c52d8f4fe" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 38627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "61623a6e3f344baea8a5a811a4fd68f9" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d8d873a93f2422489d8599536b01df2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35999 } health_report { overall_health: UNKNOWN } } }
I20260929 03:09:37.210342 27716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.206968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38468 (local address 127.7.209.66:38627)
0929 03:09:37.207099 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:37.207103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:37.207117 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:09:37.207233 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:37.207236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:37.207305 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:37.207405 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:09:37.207620 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.208168 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.208982 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.209194 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.209261 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.209509 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:09:37.210019 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:09:37.210112 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:37.210139 (+ 27us) server_negotiation.cc:299] Negotiation successful
0929 03:09:37.210193 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 03:09:37.216935 27387 raft_consensus.cc:1261] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 61623a6e3f344baea8a5a811a4fd68f9 [term 1 FOLLOWER]: Refusing update from remote peer 0e0cfd4ca56c4e87b02c011c52d8f4fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:09:37.216934 27655 raft_consensus.cc:1261] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2 [term 1 FOLLOWER]: Refusing update from remote peer 0e0cfd4ca56c4e87b02c011c52d8f4fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:09:37.217684 27728 consensus_queue.cc:1035] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "61623a6e3f344baea8a5a811a4fd68f9" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 33157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:09:37.217864 27725 consensus_queue.cc:1035] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d8d873a93f2422489d8599536b01df2" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:09:37.225773 27733 mvcc.cc:205] Tried to move back new op lower bound from 7334508041072697344 to 7334508040899264512. Current Snapshot: MvccSnapshot[applied={T|T < 7334508041072697344}]
I20260929 03:09:37.236574 27240 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:09:37.236835 27240 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:09:37.239781 27240 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efea5aaa7a0641dbbab63c4a662e79cf: Sending DeleteTablet for 3 replicas of tablet c6b4e5cbd3b54f63bfb338b11ec0f750
I20260929 03:09:37.240764 27501 tablet_service.cc:1514] Processing DeleteTablet for tablet c6b4e5cbd3b54f63bfb338b11ec0f750 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:37 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:38456
I20260929 03:09:37.241029 27635 tablet_service.cc:1514] Processing DeleteTablet for tablet c6b4e5cbd3b54f63bfb338b11ec0f750 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:37 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:42308
I20260929 03:09:37.241446 27367 tablet_service.cc:1514] Processing DeleteTablet for tablet c6b4e5cbd3b54f63bfb338b11ec0f750 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:37 UTC) from {username='kudu', principal='kudu/127.7.209.126@KRBTEST.COM'} at 127.0.0.1:56706
I20260929 03:09:37.241911 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 27304
I20260929 03:09:37.241941 27740 tablet_replica.cc:330] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe: stopping tablet replica
I20260929 03:09:37.242852 27740 raft_consensus.cc:2229] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:09:37.243425 27740 raft_consensus.cc:2258] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:37.245267 27740 ts_tablet_manager.cc:1905] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:09:37.245869 27741 tablet_replica.cc:330] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2: stopping tablet replica
I20260929 03:09:37.246323 27741 raft_consensus.cc:2229] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:09:37.246788 27741 raft_consensus.cc:2258] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:37.248492 27741 ts_tablet_manager.cc:1905] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:09:37.249208 27740 ts_tablet_manager.cc:1918] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:09:37.249323 27740 log.cc:1196] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/c6b4e5cbd3b54f63bfb338b11ec0f750
I20260929 03:09:37.249699 27740 ts_tablet_manager.cc:1939] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 0e0cfd4ca56c4e87b02c011c52d8f4fe: Deleting consensus metadata
W20260929 03:09:37.250958 27226 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:33157 (error 108)
I20260929 03:09:37.251111 27227 catalog_manager.cc:4841] TS 0e0cfd4ca56c4e87b02c011c52d8f4fe (127.7.209.66:38627): tablet c6b4e5cbd3b54f63bfb338b11ec0f750 (table test-table [id=d0dae62a2fa84452be6f8bf648989291]) successfully deleted
I20260929 03:09:37.251915 27741 ts_tablet_manager.cc:1918] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:09:37.251976 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 27438
I20260929 03:09:37.252229 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.251878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:33157 (local address 127.0.0.1:56714)
0929 03:09:37.252068 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:37.252136 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:33157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260929 03:09:37.253021 27226 catalog_manager.cc:4584] TS 61623a6e3f344baea8a5a811a4fd68f9 (127.7.209.65:33157): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c6b4e5cbd3b54f63bfb338b11ec0f750: Network error: Client connection negotiation failed: client connection to 127.7.209.65:33157: connect: Connection refused (error 111)
I20260929 03:09:37.253572 27741 log.cc:1196] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/c6b4e5cbd3b54f63bfb338b11ec0f750
I20260929 03:09:37.253940 27741 ts_tablet_manager.cc:1939] T c6b4e5cbd3b54f63bfb338b11ec0f750 P 6d8d873a93f2422489d8599536b01df2: Deleting consensus metadata
I20260929 03:09:37.255244 27228 catalog_manager.cc:4841] TS 6d8d873a93f2422489d8599536b01df2 (127.7.209.67:35999): tablet c6b4e5cbd3b54f63bfb338b11ec0f750 (table test-table [id=d0dae62a2fa84452be6f8bf648989291]) successfully deleted
I20260929 03:09:37.258288 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 27572
I20260929 03:09:37.263747 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 27210
2026-09-29T03:09:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2348 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T03:09:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:37Z Disabled control of system clock
I20260929 03:09:37.296924 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:33137
--webserver_interface=127.7.209.126
--webserver_port=0
--builtin_ntp_servers=127.7.209.84:42589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:33137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 03:09:37.430706 27747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:37.431077 27747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:37.431115 27747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:37.435483 27747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:37.435567 27747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:37.435585 27747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:37.435602 27747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:37.435616 27747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:37.440855 27747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:42589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:33137
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:33137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:37.442555 27747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:37.444520 27747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:09:37.454352 27747 server_base.cc:1034] running on GCE node
W20260929 03:09:37.454299 27753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:37.454298 27755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:37.454312 27752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:37.455219 27747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:37.456564 27747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:37.457698 27747 hybrid_clock.cc:648] HybridClock initialized: now 1790651377457678 us; error 52 us; skew 500 ppm
I20260929 03:09:37.459794 27747 webserver.cc:460] Webserver started at http://127.7.209.126:36711/ using document root <none> and password file <none>
I20260929 03:09:37.460450 27747 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:37.460520 27747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:37.460748 27747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:37.463163 27747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "563e3c3bd2884de39e6c330f6b304014"
format_stamp: "Formatted at 2026-09-29 03:09:37 on dist-test-slave-bdps"
server_key: "c73419db27523322ae05206403d51026"
server_key_iv: "4897ef124e5011ea0b59cd2d56d95e48"
server_key_version: "encryptionkey@0"
I20260929 03:09:37.463788 27747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "563e3c3bd2884de39e6c330f6b304014"
format_stamp: "Formatted at 2026-09-29 03:09:37 on dist-test-slave-bdps"
server_key: "c73419db27523322ae05206403d51026"
server_key_iv: "4897ef124e5011ea0b59cd2d56d95e48"
server_key_version: "encryptionkey@0"
I20260929 03:09:37.467976 27747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:09:37.470649 27761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:37.472091 27747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:37.472218 27747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "563e3c3bd2884de39e6c330f6b304014"
format_stamp: "Formatted at 2026-09-29 03:09:37 on dist-test-slave-bdps"
server_key: "c73419db27523322ae05206403d51026"
server_key_iv: "4897ef124e5011ea0b59cd2d56d95e48"
server_key_version: "encryptionkey@0"
I20260929 03:09:37.472329 27747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:37.489292 27747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:37.490123 27747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:37.490319 27747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:37.497967 27747 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:33137
I20260929 03:09:37.497995 27813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:33137 every 8 connection(s)
I20260929 03:09:37.499063 27747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:37.502882 27814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:37.504629 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 27747
I20260929 03:09:37.504767 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:37.505093 8005 external_mini_cluster.cc:1428] Setting key ed1e33f10d781908842f0a4e29ff3a0c
I20260929 03:09:37.510617 27814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014: Bootstrap starting.
I20260929 03:09:37.513717 27814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:37.514709 27814 log.cc:824] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:37.516508 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.506778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52108 (local address 127.7.209.126:33137)
0929 03:09:37.507528 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:37.507551 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:37.507602 (+ 51us) server_negotiation.cc:407] Connection header received
0929 03:09:37.508498 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:37.508525 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:37.509115 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:37.509539 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:37.510057 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.511539 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.512461 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.513182 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.513342 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.514231 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:37.514237 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:37.514256 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:37.514824 (+ 568us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:37.514901 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:37.514929 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:37.515027 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:37.515142 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 03:09:37.515463 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":112,"threads_started":1}
I20260929 03:09:37.517117 27814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014: No bootstrap required, opened a new log
I20260929 03:09:37.521179 27814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563e3c3bd2884de39e6c330f6b304014" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 33137 } }
I20260929 03:09:37.521431 27814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:37.521484 27814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 563e3c3bd2884de39e6c330f6b304014, State: Initialized, Role: FOLLOWER
I20260929 03:09:37.521981 27814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563e3c3bd2884de39e6c330f6b304014" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 33137 } }
I20260929 03:09:37.522116 27814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:37.522181 27814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:37.522284 27814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:37.523325 27814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563e3c3bd2884de39e6c330f6b304014" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 33137 } }
I20260929 03:09:37.523689 27814 leader_election.cc:302] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 563e3c3bd2884de39e6c330f6b304014; no voters:
I20260929 03:09:37.523985 27814 leader_election.cc:288] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:37.524089 27819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:37.524365 27819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [term 1 LEADER]: Becoming Leader. State: Replica: 563e3c3bd2884de39e6c330f6b304014, State: Running, Role: LEADER
I20260929 03:09:37.524699 27819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563e3c3bd2884de39e6c330f6b304014" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 33137 } }
I20260929 03:09:37.525324 27814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:37.526588 27821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 563e3c3bd2884de39e6c330f6b304014. Latest consensus state: current_term: 1 leader_uuid: "563e3c3bd2884de39e6c330f6b304014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563e3c3bd2884de39e6c330f6b304014" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 33137 } } }
I20260929 03:09:37.526588 27820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "563e3c3bd2884de39e6c330f6b304014" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "563e3c3bd2884de39e6c330f6b304014" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 33137 } } }
I20260929 03:09:37.526750 27820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:37.526750 27821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:37.527220 27828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:37.530764 27828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:37.537088 27828 catalog_manager.cc:1347] Generated new cluster ID: 6de1ec13f40e43da9e465fa600be7d54
I20260929 03:09:37.537161 27828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:37.567287 27828 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:37.568435 27828 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:37.575019 27828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014: Generated new TSK 0
I20260929 03:09:37.575820 27828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:37.587829 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.65:0
--local_ip_for_outbound_sockets=127.7.209.65
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:33137
--builtin_ntp_servers=127.7.209.84:42589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 03:09:37.716600 27838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:37.716925 27838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:37.716969 27838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:37.721026 27838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:37.721081 27838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:37.721151 27838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.65
I20260929 03:09:37.726322 27838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:42589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.209.65
--webserver_port=0
--tserver_master_addrs=127.7.209.126:33137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.65
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:37.727591 27838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:37.729271 27838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:37.739874 27846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:37.739957 27838 server_base.cc:1034] running on GCE node
W20260929 03:09:37.739910 27843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:37.739907 27844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:37.740624 27838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:37.741307 27838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:37.742467 27838 hybrid_clock.cc:648] HybridClock initialized: now 1790651377742447 us; error 35 us; skew 500 ppm
I20260929 03:09:37.744393 27838 webserver.cc:460] Webserver started at http://127.7.209.65:41005/ using document root <none> and password file <none>
I20260929 03:09:37.745041 27838 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:37.745105 27838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:37.745335 27838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:37.747761 27838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/data/instance:
uuid: "8b21667f149a458a9d1564de6a9b9433"
format_stamp: "Formatted at 2026-09-29 03:09:37 on dist-test-slave-bdps"
server_key: "77c9048b362ecf0cf22541ee6d3d85cd"
server_key_iv: "e8d527b2d3ab6f94d7ad439484140dca"
server_key_version: "encryptionkey@0"
I20260929 03:09:37.748355 27838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance:
uuid: "8b21667f149a458a9d1564de6a9b9433"
format_stamp: "Formatted at 2026-09-29 03:09:37 on dist-test-slave-bdps"
server_key: "77c9048b362ecf0cf22541ee6d3d85cd"
server_key_iv: "e8d527b2d3ab6f94d7ad439484140dca"
server_key_version: "encryptionkey@0"
I20260929 03:09:37.752606 27838 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 03:09:37.755520 27852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:37.756965 27838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:09:37.757076 27838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/wal
uuid: "8b21667f149a458a9d1564de6a9b9433"
format_stamp: "Formatted at 2026-09-29 03:09:37 on dist-test-slave-bdps"
server_key: "77c9048b362ecf0cf22541ee6d3d85cd"
server_key_iv: "e8d527b2d3ab6f94d7ad439484140dca"
server_key_version: "encryptionkey@0"
I20260929 03:09:37.757194 27838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:37.775012 27838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:37.775776 27838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:37.775955 27838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:37.776623 27838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:37.777745 27838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:37.777793 27838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:37.777841 27838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:37.777878 27838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:37.787520 27838 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.65:43815
I20260929 03:09:37.787545 27965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.65:43815 every 8 connection(s)
I20260929 03:09:37.788607 27838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/data/info.pb
I20260929 03:09:37.796252 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 27838
I20260929 03:09:37.796367 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-0/wal/instance
I20260929 03:09:37.796686 8005 external_mini_cluster.cc:1428] Setting key 5de32ea11c04e526d80f6bc44717afe7
I20260929 03:09:37.799031 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.66:0
--local_ip_for_outbound_sockets=127.7.209.66
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:33137
--builtin_ntp_servers=127.7.209.84:42589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 03:09:37.799423 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.790845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:48435 (local address 127.7.209.126:33137)
0929 03:09:37.791014 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:37.791018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:37.792307 (+ 1289us) server_negotiation.cc:407] Connection header received
0929 03:09:37.793988 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:37.793993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:37.794072 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:37.794157 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:37.795255 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.795860 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.797171 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.797468 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.797576 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.798927 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:37.798930 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:37.798932 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:37.798945 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:37.798968 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:37.798976 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:37.799037 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:37.799208 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 03:09:37.799293 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 03:09:37.800768 27968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:37.791374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:33137 (local address 127.7.209.65:48435)
0929 03:09:37.792131 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:37.792179 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:37.793720 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:37.794370 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:37.794377 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:37.794486 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:37.795095 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.795110 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.795987 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.795990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.797048 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:37.797058 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:37.797679 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:37.797684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:37.797905 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:37.798639 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:09:37.798641 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:37.798690 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:09:37.798780 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:37.799046 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:37.799056 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:37.799062 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 03:09:37.799583 (+ 521us) client_negotiation.cc:240] Negotiation successful
0929 03:09:37.799894 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":120,"threads_started":1}
I20260929 03:09:37.802064 27966 heartbeater.cc:347] Connected to a master server at 127.7.209.126:33137
I20260929 03:09:37.802340 27966 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:37.802920 27966 heartbeater.cc:511] Master 127.7.209.126:33137 requested a full tablet report, sending...
I20260929 03:09:37.804798 27778 ts_manager.cc:194] Registered new tserver with Master: 8b21667f149a458a9d1564de6a9b9433 (127.7.209.65:43815)
I20260929 03:09:37.806675 27778 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.209.65:48435
W20260929 03:09:37.923221 27969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:37.923616 27969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:37.923662 27969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:37.928169 27969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:37.928251 27969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:37.928313 27969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.66
I20260929 03:09:37.933629 27969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:42589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.209.66
--webserver_port=0
--tserver_master_addrs=127.7.209.126:33137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.66
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:37.934937 27969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:37.936662 27969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:37.947300 27974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:37.947348 27977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:37.947347 27975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:37.947391 27969 server_base.cc:1034] running on GCE node
I20260929 03:09:37.947955 27969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:37.948608 27969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:37.949754 27969 hybrid_clock.cc:648] HybridClock initialized: now 1790651377949728 us; error 45 us; skew 500 ppm
I20260929 03:09:37.955893 27969 webserver.cc:460] Webserver started at http://127.7.209.66:39969/ using document root <none> and password file <none>
I20260929 03:09:37.956679 27969 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:37.956758 27969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:37.956992 27969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:37.959630 27969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/data/instance:
uuid: "9436b2b5874e460e985417deeb5d01e6"
format_stamp: "Formatted at 2026-09-29 03:09:37 on dist-test-slave-bdps"
server_key: "4281fc6d0ac0307f7c0de14bf100710d"
server_key_iv: "30de033ce9864e3ce2a34b494c375441"
server_key_version: "encryptionkey@0"
I20260929 03:09:37.960425 27969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance:
uuid: "9436b2b5874e460e985417deeb5d01e6"
format_stamp: "Formatted at 2026-09-29 03:09:37 on dist-test-slave-bdps"
server_key: "4281fc6d0ac0307f7c0de14bf100710d"
server_key_iv: "30de033ce9864e3ce2a34b494c375441"
server_key_version: "encryptionkey@0"
I20260929 03:09:37.964903 27969 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260929 03:09:37.967686 27983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:37.969106 27969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 03:09:37.969249 27969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/wal
uuid: "9436b2b5874e460e985417deeb5d01e6"
format_stamp: "Formatted at 2026-09-29 03:09:37 on dist-test-slave-bdps"
server_key: "4281fc6d0ac0307f7c0de14bf100710d"
server_key_iv: "30de033ce9864e3ce2a34b494c375441"
server_key_version: "encryptionkey@0"
I20260929 03:09:37.969352 27969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:37.985225 27969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:37.985961 27969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:37.986140 27969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:37.986784 27969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:37.987866 27969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:37.987914 27969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:37.987972 27969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:37.987999 27969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:37.997774 27969 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.66:33479
I20260929 03:09:37.997797 28096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.66:33479 every 8 connection(s)
I20260929 03:09:37.998845 27969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/data/info.pb
I20260929 03:09:38.007128 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 27969
I20260929 03:09:38.007298 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/wal/instance
I20260929 03:09:38.007643 8005 external_mini_cluster.cc:1428] Setting key 68abd64720ea1a555627cb61db2a5b27
I20260929 03:09:38.009490 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.000947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.66:37215 (local address 127.7.209.126:33137)
0929 03:09:38.001102 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:38.001106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:38.002382 (+ 1276us) server_negotiation.cc:407] Connection header received
0929 03:09:38.003930 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:38.003934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:38.003993 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:38.004078 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:38.005139 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.005883 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.007261 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.007556 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.007645 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.008964 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:38.008968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:38.008970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:38.008984 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:38.009010 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:38.009020 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:38.009092 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:38.009268 (+ 176us) server_negotiation.cc:299] Negotiation successful
0929 03:09:38.009342 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 03:09:38.010108 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.209.67:0
--local_ip_for_outbound_sockets=127.7.209.67
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:33137
--builtin_ntp_servers=127.7.209.84:42589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 03:09:38.010708 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.001431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:33137 (local address 127.7.209.66:37215)
0929 03:09:38.002200 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:38.002248 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:38.003669 (+ 1421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:38.004268 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:38.004276 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:38.004379 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:38.004992 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.005007 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.006002 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.006005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.007116 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.007129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.007731 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.007744 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.007967 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.008672 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:09:38.008674 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:38.008724 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:09:38.008812 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:38.009111 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:38.009120 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:38.009127 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 03:09:38.009252 (+ 125us) client_negotiation.cc:240] Negotiation successful
0929 03:09:38.009563 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":135,"threads_started":1}
I20260929 03:09:38.011873 28097 heartbeater.cc:347] Connected to a master server at 127.7.209.126:33137
I20260929 03:09:38.012151 28097 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:38.012727 28097 heartbeater.cc:511] Master 127.7.209.126:33137 requested a full tablet report, sending...
I20260929 03:09:38.013859 27778 ts_manager.cc:194] Registered new tserver with Master: 9436b2b5874e460e985417deeb5d01e6 (127.7.209.66:33479)
I20260929 03:09:38.014461 27778 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.209.66:37215
W20260929 03:09:38.134742 28100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:38.135011 28100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:38.135062 28100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:38.139142 28100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:38.139190 28100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:38.139263 28100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.209.67
I20260929 03:09:38.144023 28100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:42589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.209.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.209.67
--webserver_port=0
--tserver_master_addrs=127.7.209.126:33137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.209.67
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:38.145136 28100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:38.146651 28100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:38.158535 28105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:38.158540 28106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:38.158687 28108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:38.159480 28100 server_base.cc:1034] running on GCE node
I20260929 03:09:38.159934 28100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:38.160653 28100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:38.161813 28100 hybrid_clock.cc:648] HybridClock initialized: now 1790651378161771 us; error 55 us; skew 500 ppm
I20260929 03:09:38.164075 28100 webserver.cc:460] Webserver started at http://127.7.209.67:44861/ using document root <none> and password file <none>
I20260929 03:09:38.164762 28100 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:38.164835 28100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:38.165074 28100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:38.167600 28100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/data/instance:
uuid: "80f09e44a88c4761b32bbcf3f427da54"
format_stamp: "Formatted at 2026-09-29 03:09:38 on dist-test-slave-bdps"
server_key: "a4cb9ffe2335a87f2cb516d5f29eb5eb"
server_key_iv: "d019531115a3a5765591918009277be6"
server_key_version: "encryptionkey@0"
I20260929 03:09:38.168220 28100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance:
uuid: "80f09e44a88c4761b32bbcf3f427da54"
format_stamp: "Formatted at 2026-09-29 03:09:38 on dist-test-slave-bdps"
server_key: "a4cb9ffe2335a87f2cb516d5f29eb5eb"
server_key_iv: "d019531115a3a5765591918009277be6"
server_key_version: "encryptionkey@0"
I20260929 03:09:38.172412 28100 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 03:09:38.175257 28114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:38.176725 28100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:09:38.176846 28100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/wal
uuid: "80f09e44a88c4761b32bbcf3f427da54"
format_stamp: "Formatted at 2026-09-29 03:09:38 on dist-test-slave-bdps"
server_key: "a4cb9ffe2335a87f2cb516d5f29eb5eb"
server_key_iv: "d019531115a3a5765591918009277be6"
server_key_version: "encryptionkey@0"
I20260929 03:09:38.176955 28100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:38.202582 28100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:38.203513 28100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:38.203743 28100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:38.204522 28100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 03:09:38.205715 28100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 03:09:38.205776 28100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:38.205835 28100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 03:09:38.205866 28100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:38.215788 28100 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.67:35465
I20260929 03:09:38.215816 28227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.67:35465 every 8 connection(s)
I20260929 03:09:38.216840 28100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/data/info.pb
I20260929 03:09:38.218307 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 28100
I20260929 03:09:38.218430 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/wal/instance
I20260929 03:09:38.218762 8005 external_mini_cluster.cc:1428] Setting key 8ee1b5d4091f8255069f3cffd8b49fc1
I20260929 03:09:38.227310 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.219165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.67:39769 (local address 127.7.209.126:33137)
0929 03:09:38.219328 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:38.219333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:38.220788 (+ 1455us) server_negotiation.cc:407] Connection header received
0929 03:09:38.222402 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:38.222405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:38.222460 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:38.222524 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:38.223559 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.224147 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.225328 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.225573 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.225700 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.226820 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:38.226823 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:38.226826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:38.226838 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:38.226860 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:38.226868 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:38.226945 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:38.227115 (+ 170us) server_negotiation.cc:299] Negotiation successful
0929 03:09:38.227173 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 03:09:38.228319 28230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.219716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:33137 (local address 127.7.209.67:39769)
0929 03:09:38.220606 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:38.220667 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:38.222167 (+ 1500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:38.222718 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:38.222726 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:38.222832 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:09:38.223432 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.223447 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.224257 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.224261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.225231 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.225240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.225742 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.225745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.225920 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.226532 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 03:09:38.226533 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:09:38.226578 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 03:09:38.226664 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:09:38.226978 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:09:38.226988 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:09:38.226996 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 03:09:38.227117 (+ 121us) client_negotiation.cc:240] Negotiation successful
0929 03:09:38.227376 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":142,"threads_started":1}
I20260929 03:09:38.229404 28228 heartbeater.cc:347] Connected to a master server at 127.7.209.126:33137
I20260929 03:09:38.229651 28228 heartbeater.cc:464] Registering TS with master...
I20260929 03:09:38.230198 28228 heartbeater.cc:511] Master 127.7.209.126:33137 requested a full tablet report, sending...
I20260929 03:09:38.231285 27778 ts_manager.cc:194] Registered new tserver with Master: 80f09e44a88c4761b32bbcf3f427da54 (127.7.209.67:35465)
I20260929 03:09:38.232046 27778 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.209.67:39769
I20260929 03:09:38.232789 8005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 03:09:38.239210 27817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.235204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52112 (local address 127.7.209.126:33137)
0929 03:09:38.235331 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:38.235336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:38.235597 (+ 261us) server_negotiation.cc:407] Connection header received
0929 03:09:38.235732 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:38.235735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:38.235782 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:38.235853 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:38.236172 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.236780 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.237662 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.237884 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.237971 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.238797 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:38.238801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:38.238804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:38.238817 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:38.238837 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:38.238850 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:38.238908 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:38.239010 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 03:09:38.239064 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 03:09:38.242058 27778 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 03:09:38.245453 27778 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 03:09:38.261062 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.254931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38330 (local address 127.7.209.67:35465)
0929 03:09:38.255779 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:38.255794 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:38.256320 (+ 526us) server_negotiation.cc:407] Connection header received
0929 03:09:38.256399 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:38.256403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:38.256547 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:38.256688 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:38.257025 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.258254 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.259434 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.260345 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.260465 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.260735 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:38.260790 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 03:09:38.260897 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":178,"threads_started":1}
I20260929 03:09:38.261060 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.254704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:33479 (local address 127.0.0.1:51970)
0929 03:09:38.255831 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:38.255890 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:38.256029 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:38.256600 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:38.256606 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:38.256628 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:38.256775 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.256782 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.257982 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.257985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.259462 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.259472 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.260526 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.260530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.260618 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.260639 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 03:09:38.260704 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 03:09:38.260861 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":49,"spinlock_wait_cycles":677248,"thread_start_us":128,"threads_started":1}
I20260929 03:09:38.261641 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.255027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:43815 (local address 127.0.0.1:41292)
0929 03:09:38.256011 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:38.256026 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:38.256118 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:38.256846 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:38.256848 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:38.256866 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:38.256965 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.256970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.258623 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.258626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.259699 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.259706 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.260760 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.260763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.260846 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.260866 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 03:09:38.261442 (+ 576us) client_negotiation.cc:240] Negotiation successful
0929 03:09:38.261511 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"thread_start_us":92,"threads_started":1}
I20260929 03:09:38.261708 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.255552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51970 (local address 127.7.209.66:33479)
0929 03:09:38.256022 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:38.256035 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:38.256063 (+ 28us) server_negotiation.cc:407] Connection header received
0929 03:09:38.256185 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:38.256192 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:38.256450 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:38.256606 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:38.256883 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.257861 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.259583 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.260405 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.261135 (+ 730us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.261348 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:38.261402 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 03:09:38.261506 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":137,"threads_started":1}
I20260929 03:09:38.261058 28242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.254599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:35465 (local address 127.0.0.1:38330)
0929 03:09:38.256208 (+ 1609us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:38.256221 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:38.256309 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:38.256801 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:38.256804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:38.256820 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:38.256912 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.256917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.258378 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.258381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.259308 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.259318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.260471 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.260475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.260579 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.260615 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 03:09:38.260716 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 03:09:38.260869 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":74,"thread_start_us":84,"threads_started":1}
I20260929 03:09:38.262185 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.255833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41292 (local address 127.7.209.65:43815)
0929 03:09:38.256223 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:38.256228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:38.256249 (+ 21us) server_negotiation.cc:407] Connection header received
0929 03:09:38.256330 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:38.256335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:38.256524 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:38.256688 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:38.257093 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.258511 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.259799 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.260390 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.260485 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.260689 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:38.261916 (+ 1227us) server_negotiation.cc:299] Negotiation successful
0929 03:09:38.262018 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":76,"threads_started":1}
I20260929 03:09:38.264948 27900 tablet_service.cc:1467] Processing CreateTablet for tablet b6df77a8317f4c30bc441e671a47f444 (DEFAULT_TABLE table=test-table [id=3aaea14a2cc34ffab11e68748e775817]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:38.264950 28031 tablet_service.cc:1467] Processing CreateTablet for tablet b6df77a8317f4c30bc441e671a47f444 (DEFAULT_TABLE table=test-table [id=3aaea14a2cc34ffab11e68748e775817]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:38.264974 28162 tablet_service.cc:1467] Processing CreateTablet for tablet b6df77a8317f4c30bc441e671a47f444 (DEFAULT_TABLE table=test-table [id=3aaea14a2cc34ffab11e68748e775817]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 03:09:38.266242 28162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6df77a8317f4c30bc441e671a47f444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:38.266242 28031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6df77a8317f4c30bc441e671a47f444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:38.266248 27900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6df77a8317f4c30bc441e671a47f444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:38.274222 28245 tablet_bootstrap.cc:492] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54: Bootstrap starting.
I20260929 03:09:38.275197 28246 tablet_bootstrap.cc:492] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433: Bootstrap starting.
I20260929 03:09:38.275969 28247 tablet_bootstrap.cc:492] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6: Bootstrap starting.
I20260929 03:09:38.277307 28245 tablet_bootstrap.cc:654] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:38.277709 28246 tablet_bootstrap.cc:654] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:38.278256 28245 log.cc:824] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:38.278576 28247 tablet_bootstrap.cc:654] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:38.278585 28246 log.cc:824] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:38.279407 28247 log.cc:824] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:38.280565 28246 tablet_bootstrap.cc:492] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433: No bootstrap required, opened a new log
I20260929 03:09:38.280582 28245 tablet_bootstrap.cc:492] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54: No bootstrap required, opened a new log
I20260929 03:09:38.280772 28246 ts_tablet_manager.cc:1397] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:09:38.280784 28245 ts_tablet_manager.cc:1397] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260929 03:09:38.281323 28247 tablet_bootstrap.cc:492] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6: No bootstrap required, opened a new log
I20260929 03:09:38.281487 28247 ts_tablet_manager.cc:1397] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260929 03:09:38.284737 28246 raft_consensus.cc:348] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f09e44a88c4761b32bbcf3f427da54" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35465 } } peers { permanent_uuid: "9436b2b5874e460e985417deeb5d01e6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33479 } } peers { permanent_uuid: "8b21667f149a458a9d1564de6a9b9433" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 43815 } }
I20260929 03:09:38.284977 28246 raft_consensus.cc:374] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:38.285039 28246 raft_consensus.cc:729] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b21667f149a458a9d1564de6a9b9433, State: Initialized, Role: FOLLOWER
I20260929 03:09:38.285219 28245 raft_consensus.cc:348] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f09e44a88c4761b32bbcf3f427da54" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35465 } } peers { permanent_uuid: "9436b2b5874e460e985417deeb5d01e6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33479 } } peers { permanent_uuid: "8b21667f149a458a9d1564de6a9b9433" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 43815 } }
I20260929 03:09:38.285502 28245 raft_consensus.cc:374] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:38.285557 28245 raft_consensus.cc:729] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80f09e44a88c4761b32bbcf3f427da54, State: Initialized, Role: FOLLOWER
I20260929 03:09:38.285493 28246 consensus_queue.cc:260] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f09e44a88c4761b32bbcf3f427da54" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35465 } } peers { permanent_uuid: "9436b2b5874e460e985417deeb5d01e6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33479 } } peers { permanent_uuid: "8b21667f149a458a9d1564de6a9b9433" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 43815 } }
I20260929 03:09:38.285576 28247 raft_consensus.cc:348] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f09e44a88c4761b32bbcf3f427da54" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35465 } } peers { permanent_uuid: "9436b2b5874e460e985417deeb5d01e6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33479 } } peers { permanent_uuid: "8b21667f149a458a9d1564de6a9b9433" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 43815 } }
I20260929 03:09:38.285861 28247 raft_consensus.cc:374] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:38.285923 28247 raft_consensus.cc:729] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9436b2b5874e460e985417deeb5d01e6, State: Initialized, Role: FOLLOWER
I20260929 03:09:38.285997 28245 consensus_queue.cc:260] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f09e44a88c4761b32bbcf3f427da54" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35465 } } peers { permanent_uuid: "9436b2b5874e460e985417deeb5d01e6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33479 } } peers { permanent_uuid: "8b21667f149a458a9d1564de6a9b9433" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 43815 } }
I20260929 03:09:38.286320 28247 consensus_queue.cc:260] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f09e44a88c4761b32bbcf3f427da54" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35465 } } peers { permanent_uuid: "9436b2b5874e460e985417deeb5d01e6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33479 } } peers { permanent_uuid: "8b21667f149a458a9d1564de6a9b9433" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 43815 } }
I20260929 03:09:38.286595 28246 ts_tablet_manager.cc:1428] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 03:09:38.286782 28228 heartbeater.cc:503] Master 127.7.209.126:33137 was elected leader, sending a full tablet report...
I20260929 03:09:38.286936 27966 heartbeater.cc:503] Master 127.7.209.126:33137 was elected leader, sending a full tablet report...
I20260929 03:09:38.287042 28245 ts_tablet_manager.cc:1428] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260929 03:09:38.287165 28247 ts_tablet_manager.cc:1428] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260929 03:09:38.287487 28097 heartbeater.cc:503] Master 127.7.209.126:33137 was elected leader, sending a full tablet report...
W20260929 03:09:38.289773 27967 tablet.cc:2369] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 03:09:38.415866 28251 raft_consensus.cc:482] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 03:09:38.416136 28251 raft_consensus.cc:504] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f09e44a88c4761b32bbcf3f427da54" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35465 } } peers { permanent_uuid: "9436b2b5874e460e985417deeb5d01e6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33479 } } peers { permanent_uuid: "8b21667f149a458a9d1564de6a9b9433" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 43815 } }
I20260929 03:09:38.417522 28251 leader_election.cc:288] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80f09e44a88c4761b32bbcf3f427da54 (127.7.209.67:35465), 9436b2b5874e460e985417deeb5d01e6 (127.7.209.66:33479)
I20260929 03:09:38.422964 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.417979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:39551 (local address 127.7.209.66:33479)
0929 03:09:38.418141 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:38.418146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:38.418346 (+ 200us) server_negotiation.cc:407] Connection header received
0929 03:09:38.418515 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:38.418518 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:38.418582 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:38.418678 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:38.419051 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.419886 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.421651 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.422291 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.422411 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.422468 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:38.422727 (+ 259us) server_negotiation.cc:299] Negotiation successful
0929 03:09:38.422809 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 03:09:38.422964 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.417978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.209.65:46115 (local address 127.7.209.67:35465)
0929 03:09:38.418133 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:38.418138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:38.418411 (+ 273us) server_negotiation.cc:407] Connection header received
0929 03:09:38.418537 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:38.418540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:38.418587 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:38.418678 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 03:09:38.419051 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.419886 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.421651 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.422283 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.422411 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.422468 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:38.422727 (+ 259us) server_negotiation.cc:299] Negotiation successful
0929 03:09:38.422822 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 03:09:38.423026 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.417831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:33479 (local address 127.7.209.65:39551)
0929 03:09:38.418240 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:38.418257 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:38.418403 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:38.418694 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:38.418697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:38.418721 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:38.418937 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.418944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.420005 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.420009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.421497 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.421510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.422401 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.422404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.422592 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.422630 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 03:09:38.422707 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 03:09:38.422816 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":167,"spinlock_wait_cycles":1792,"thread_start_us":109,"threads_started":1}
I20260929 03:09:38.423026 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.417831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.67:35465 (local address 127.7.209.65:46115)
0929 03:09:38.418332 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:38.418343 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0929 03:09:38.418427 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:09:38.418694 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:09:38.418697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:09:38.418726 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 03:09:38.418939 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.418944 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.420005 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.420009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.421497 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:09:38.421509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.422401 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.422404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:09:38.422588 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.422630 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 03:09:38.422707 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 03:09:38.422816 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":248,"spinlock_wait_cycles":7296,"thread_start_us":144,"threads_started":1}
I20260929 03:09:38.423830 28182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6df77a8317f4c30bc441e671a47f444" candidate_uuid: "8b21667f149a458a9d1564de6a9b9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80f09e44a88c4761b32bbcf3f427da54" is_pre_election: true
I20260929 03:09:38.423969 28051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6df77a8317f4c30bc441e671a47f444" candidate_uuid: "8b21667f149a458a9d1564de6a9b9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9436b2b5874e460e985417deeb5d01e6" is_pre_election: true
I20260929 03:09:38.424187 28182 raft_consensus.cc:2454] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b21667f149a458a9d1564de6a9b9433 in term 0.
I20260929 03:09:38.424252 28051 raft_consensus.cc:2454] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b21667f149a458a9d1564de6a9b9433 in term 0.
I20260929 03:09:38.424718 27854 leader_election.cc:302] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80f09e44a88c4761b32bbcf3f427da54, 8b21667f149a458a9d1564de6a9b9433; no voters:
I20260929 03:09:38.424999 28251 raft_consensus.cc:2781] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 03:09:38.425100 28251 raft_consensus.cc:482] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 03:09:38.425144 28251 raft_consensus.cc:3037] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:38.426561 28251 raft_consensus.cc:504] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f09e44a88c4761b32bbcf3f427da54" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35465 } } peers { permanent_uuid: "9436b2b5874e460e985417deeb5d01e6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33479 } } peers { permanent_uuid: "8b21667f149a458a9d1564de6a9b9433" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 43815 } }
I20260929 03:09:38.426884 28251 leader_election.cc:288] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [CANDIDATE]: Term 1 election: Requested vote from peers 80f09e44a88c4761b32bbcf3f427da54 (127.7.209.67:35465), 9436b2b5874e460e985417deeb5d01e6 (127.7.209.66:33479)
I20260929 03:09:38.427207 28182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6df77a8317f4c30bc441e671a47f444" candidate_uuid: "8b21667f149a458a9d1564de6a9b9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80f09e44a88c4761b32bbcf3f427da54"
I20260929 03:09:38.427209 28051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6df77a8317f4c30bc441e671a47f444" candidate_uuid: "8b21667f149a458a9d1564de6a9b9433" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9436b2b5874e460e985417deeb5d01e6"
I20260929 03:09:38.427340 28182 raft_consensus.cc:3037] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:38.427342 28051 raft_consensus.cc:3037] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:38.428483 28182 raft_consensus.cc:2454] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b21667f149a458a9d1564de6a9b9433 in term 1.
I20260929 03:09:38.428735 28051 raft_consensus.cc:2454] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b21667f149a458a9d1564de6a9b9433 in term 1.
I20260929 03:09:38.428858 27854 leader_election.cc:302] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80f09e44a88c4761b32bbcf3f427da54, 8b21667f149a458a9d1564de6a9b9433; no voters:
I20260929 03:09:38.429039 28251 raft_consensus.cc:2781] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:38.429283 28251 raft_consensus.cc:686] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 1 LEADER]: Becoming Leader. State: Replica: 8b21667f149a458a9d1564de6a9b9433, State: Running, Role: LEADER
I20260929 03:09:38.429672 28251 consensus_queue.cc:237] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f09e44a88c4761b32bbcf3f427da54" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35465 } } peers { permanent_uuid: "9436b2b5874e460e985417deeb5d01e6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33479 } } peers { permanent_uuid: "8b21667f149a458a9d1564de6a9b9433" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 43815 } }
I20260929 03:09:38.433274 27776 catalog_manager.cc:5495] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b21667f149a458a9d1564de6a9b9433 (127.7.209.65). New cstate: current_term: 1 leader_uuid: "8b21667f149a458a9d1564de6a9b9433" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80f09e44a88c4761b32bbcf3f427da54" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9436b2b5874e460e985417deeb5d01e6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b21667f149a458a9d1564de6a9b9433" member_type: VOTER last_known_addr { host: "127.7.209.65" port: 43815 } health_report { overall_health: HEALTHY } } }
I20260929 03:09:38.450524 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.446890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.7.209.65:43815)
0929 03:09:38.447021 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:38.447025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:38.447038 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:09:38.447167 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:38.447169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:38.447235 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:38.447330 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:09:38.447524 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.448083 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.448918 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:38.449128 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:38.449199 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:38.449445 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:09:38.450132 (+ 687us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:09:38.450239 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:38.450297 (+ 58us) server_negotiation.cc:299] Negotiation successful
0929 03:09:38.450356 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:09:38.457485 28051 raft_consensus.cc:1261] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6 [term 1 FOLLOWER]: Refusing update from remote peer 8b21667f149a458a9d1564de6a9b9433: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:09:38.457484 28182 raft_consensus.cc:1261] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54 [term 1 FOLLOWER]: Refusing update from remote peer 8b21667f149a458a9d1564de6a9b9433: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:09:38.458228 28256 consensus_queue.cc:1035] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80f09e44a88c4761b32bbcf3f427da54" member_type: VOTER last_known_addr { host: "127.7.209.67" port: 35465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:09:38.458431 28251 consensus_queue.cc:1035] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9436b2b5874e460e985417deeb5d01e6" member_type: VOTER last_known_addr { host: "127.7.209.66" port: 33479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:09:38.467011 28261 mvcc.cc:205] Tried to move back new op lower bound from 7334508046153646080 to 7334508046050848768. Current Snapshot: MvccSnapshot[applied={T|T < 7334508046153646080}]
I20260929 03:09:38.467221 28260 mvcc.cc:205] Tried to move back new op lower bound from 7334508046153646080 to 7334508046050848768. Current Snapshot: MvccSnapshot[applied={T|T < 7334508046153646080}]
I20260929 03:09:38.478974 27778 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:09:38.479275 27778 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 03:09:38.482681 27778 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 563e3c3bd2884de39e6c330f6b304014: Sending DeleteTablet for 3 replicas of tablet b6df77a8317f4c30bc441e671a47f444
I20260929 03:09:38.483786 28031 tablet_service.cc:1514] Processing DeleteTablet for tablet b6df77a8317f4c30bc441e671a47f444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:38 UTC) from {username='slave'} at 127.0.0.1:51970
I20260929 03:09:38.483771 27900 tablet_service.cc:1514] Processing DeleteTablet for tablet b6df77a8317f4c30bc441e671a47f444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:38 UTC) from {username='slave'} at 127.0.0.1:41292
I20260929 03:09:38.483815 28162 tablet_service.cc:1514] Processing DeleteTablet for tablet b6df77a8317f4c30bc441e671a47f444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 03:09:38 UTC) from {username='slave'} at 127.0.0.1:38330
I20260929 03:09:38.484609 28270 tablet_replica.cc:330] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433: stopping tablet replica
I20260929 03:09:38.485069 28270 raft_consensus.cc:2229] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 1 LEADER]: Raft consensus shutting down.
I20260929 03:09:38.485281 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 27838
I20260929 03:09:38.485648 28270 raft_consensus.cc:2258] T b6df77a8317f4c30bc441e671a47f444 P 8b21667f149a458a9d1564de6a9b9433 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:38.486451 28269 tablet_replica.cc:330] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6: stopping tablet replica
I20260929 03:09:38.486846 28269 raft_consensus.cc:2229] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:09:38.487238 28269 raft_consensus.cc:2258] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:38.489578 28269 ts_tablet_manager.cc:1905] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 03:09:38.492193 28269 ts_tablet_manager.cc:1918] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:09:38.492280 28269 log.cc:1196] T b6df77a8317f4c30bc441e671a47f444 P 9436b2b5874e460e985417deeb5d01e6: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-1/wal/wals/b6df77a8317f4c30bc441e671a47f444
I20260929 03:09:38.492551 28271 tablet_replica.cc:330] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54: stopping tablet replica
I20260929 03:09:38.492862 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 27969
W20260929 03:09:38.492841 27765 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.209.65:43815 (error 108)
I20260929 03:09:38.492946 28271 raft_consensus.cc:2229] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 03:09:38.493290 28271 raft_consensus.cc:2258] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 03:09:38.495599 28242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.495297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.65:43815 (local address 127.0.0.1:41306)
0929 03:09:38.495436 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:38.495507 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.65:43815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 03:09:38.495813 28271 ts_tablet_manager.cc:1905] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260929 03:09:38.495816 27765 catalog_manager.cc:4584] TS 8b21667f149a458a9d1564de6a9b9433 (127.7.209.65:43815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6df77a8317f4c30bc441e671a47f444: Network error: Client connection negotiation failed: client connection to 127.7.209.65:43815: connect: Connection refused (error 111)
I20260929 03:09:38.498776 28271 ts_tablet_manager.cc:1918] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 03:09:38.498878 28271 log.cc:1196] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54: Deleting WAL directory at /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790651120485886-8005-0/minicluster-data/ts-2/wal/wals/b6df77a8317f4c30bc441e671a47f444
I20260929 03:09:38.499184 28271 ts_tablet_manager.cc:1939] T b6df77a8317f4c30bc441e671a47f444 P 80f09e44a88c4761b32bbcf3f427da54: Deleting consensus metadata
I20260929 03:09:38.500049 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 28100
I20260929 03:09:38.501586 28242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:38.500258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.66:33479 (local address 127.0.0.1:51974)
0929 03:09:38.501428 (+ 1170us) negotiation.cc:107] Waiting for socket to connect
0929 03:09:38.501497 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.66:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1072}
I20260929 03:09:38.505839 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 27747
2026-09-29T03:09:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1241 ms)
[----------] 6 tests from EncryptionPolicyTest (10734 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:38 dist-test-slave-bdps krb5kdc[28275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:38 dist-test-slave-bdps krb5kdc[28275](info): set up 2 sockets
Sep 29 03:09:38 dist-test-slave-bdps krb5kdc[28275](info): commencing operation
krb5kdc: starting...
W20260929 03:09:39.462455 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:39 dist-test-slave-bdps krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:09:39.592388 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33949
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:39.717192 28288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:39.717566 28288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:39.717656 28288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:39.721905 28288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:39.721987 28288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:39.722014 28288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:39.722035 28288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:39.722056 28288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:39.727773 28288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33949
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:39.729017 28288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:39.730266 28288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:39.739490 28294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:39.739499 28293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:39.739655 28288 server_base.cc:1034] running on GCE node
W20260929 03:09:39.739558 28296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:39.740406 28288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:39.741583 28288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:39.742770 28288 hybrid_clock.cc:648] HybridClock initialized: now 1790651379742752 us; error 35 us; skew 500 ppm
Sep 29 03:09:39 dist-test-slave-bdps krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651379, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:39.745594 28288 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:09:39.746558 28288 webserver.cc:460] Webserver started at http://127.0.0.1:45817/ using document root <none> and password file <none>
I20260929 03:09:39.747138 28288 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:39.747205 28288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:39.747435 28288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:39.749739 28288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "35ca1d897db84c11a3a259a108765d09"
format_stamp: "Formatted at 2026-09-29 03:09:39 on dist-test-slave-bdps"
server_key: "97d197e6d82ab4494919625827aa07f9"
server_key_iv: "7f48eba5593f70f7d8d061023c8b56b2"
server_key_version: "encryptionkey@0"
I20260929 03:09:39.750319 28288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "35ca1d897db84c11a3a259a108765d09"
format_stamp: "Formatted at 2026-09-29 03:09:39 on dist-test-slave-bdps"
server_key: "97d197e6d82ab4494919625827aa07f9"
server_key_iv: "7f48eba5593f70f7d8d061023c8b56b2"
server_key_version: "encryptionkey@0"
I20260929 03:09:39.754367 28288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:09:39.757211 28303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:39.758715 28288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:39.758849 28288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "35ca1d897db84c11a3a259a108765d09"
format_stamp: "Formatted at 2026-09-29 03:09:39 on dist-test-slave-bdps"
server_key: "97d197e6d82ab4494919625827aa07f9"
server_key_iv: "7f48eba5593f70f7d8d061023c8b56b2"
server_key_version: "encryptionkey@0"
I20260929 03:09:39.758962 28288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:39.781008 28288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:39.784248 28288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:39.784451 28288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:39.792395 28288 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33949
I20260929 03:09:39.792408 28355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33949 every 8 connection(s)
I20260929 03:09:39.793543 28288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:39.796983 28356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:39.799062 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 28288
I20260929 03:09:39.799211 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:39.799530 8005 external_mini_cluster.cc:1428] Setting key bdfbbdccf2009e63633348720d802dd3
I20260929 03:09:39.804648 28356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09: Bootstrap starting.
I20260929 03:09:39.807715 28356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:39.808751 28356 log.cc:824] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09: Log is configured to *not* fsync() on all Append() calls
Sep 29 03:09:39 dist-test-slave-bdps krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 03:09:39.811064 28356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09: No bootstrap required, opened a new log
I20260929 03:09:39.814972 28356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ca1d897db84c11a3a259a108765d09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33949 } }
I20260929 03:09:39.815163 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:39.801288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37779 (local address 127.0.0.1:33949)
0929 03:09:39.801783 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:39.801802 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:39.801847 (+ 45us) server_negotiation.cc:407] Connection header received
0929 03:09:39.802604 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:39.802625 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:39.803037 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:39.803387 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:39.804356 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:39.805438 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:39.806666 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:39.807170 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:39.807290 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:39.810433 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:39.810438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:39.810454 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:39.810481 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:39.812171 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:39.812665 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:39.812670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:39.812675 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:39.812787 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:39.813093 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:39.813096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:39.813097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:39.813519 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:39.813781 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:39.814065 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 03:09:39.814295 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":138,"threads_started":1}
I20260929 03:09:39.815240 28356 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:39.815307 28356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35ca1d897db84c11a3a259a108765d09, State: Initialized, Role: FOLLOWER
I20260929 03:09:39.815876 28356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ca1d897db84c11a3a259a108765d09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33949 } }
I20260929 03:09:39.816043 28356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:39.816103 28356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:39.816200 28356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:39.817144 28356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ca1d897db84c11a3a259a108765d09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33949 } }
I20260929 03:09:39.817543 28356 leader_election.cc:302] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35ca1d897db84c11a3a259a108765d09; no voters:
I20260929 03:09:39.817876 28356 leader_election.cc:288] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:39.817975 28361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:39.818197 28361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [term 1 LEADER]: Becoming Leader. State: Replica: 35ca1d897db84c11a3a259a108765d09, State: Running, Role: LEADER
I20260929 03:09:39.818588 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:39.818580 28361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ca1d897db84c11a3a259a108765d09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33949 } }
I20260929 03:09:39.819221 28356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:39.821312 28362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35ca1d897db84c11a3a259a108765d09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ca1d897db84c11a3a259a108765d09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33949 } } }
I20260929 03:09:39.821458 28362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:39.821779 28363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35ca1d897db84c11a3a259a108765d09. Latest consensus state: current_term: 1 leader_uuid: "35ca1d897db84c11a3a259a108765d09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ca1d897db84c11a3a259a108765d09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33949 } } }
I20260929 03:09:39.821877 28363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:39.822072 28381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:39.825634 28381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:39.830044 28359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:39.821444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39593 (local address 127.0.0.1:33949)
0929 03:09:39.821546 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:39.821551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:39.822306 (+ 755us) server_negotiation.cc:407] Connection header received
0929 03:09:39.822452 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:39.822456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:39.822503 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:39.822569 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:39.823422 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:39.823953 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:39.824891 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:39.825144 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:39.825231 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:39.826631 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:39.826635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:39.826637 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:39.826661 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:39.828118 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:39.828662 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:39.828665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:39.828667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:39.828753 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:39.829083 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:39.829086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:39.829088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:39.829289 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:39.829444 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:39.829785 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 03:09:39.829889 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 03:09:39.832309 28381 catalog_manager.cc:1347] Generated new cluster ID: 5b87b5ea516d48e68d54fb82d511e3d8
I20260929 03:09:39.832373 28381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:39.850950 28381 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:39.852015 28381 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:39.859310 28381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35ca1d897db84c11a3a259a108765d09: Generated new TSK 0
I20260929 03:09:39.860100 28381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:39.864711 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 28288
W20260929 03:09:39.871094 28273 connection.cc:535] client connection to 127.0.0.1:33949 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33949): Connection reset by peer (error 104)
2026-09-29T03:09:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1358 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:39 dist-test-slave-bdps krb5kdc[28387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:39 dist-test-slave-bdps krb5kdc[28387](info): set up 2 sockets
Sep 29 03:09:39 dist-test-slave-bdps krb5kdc[28387](info): commencing operation
krb5kdc: starting...
W20260929 03:09:40.817885 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:40 dist-test-slave-bdps krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651380, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:09:40.937637 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:41.055989 28400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:41.056300 28400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:41.056360 28400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:41.060418 28400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:41.060482 28400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:41.060510 28400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:41.060531 28400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:41.060557 28400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:41.065874 28400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40037
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:41.066993 28400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:41.068138 28400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:41.077098 28405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:41.077117 28406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:41.077200 28400 server_base.cc:1034] running on GCE node
W20260929 03:09:41.077098 28408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:41.077864 28400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:41.079119 28400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:41.080286 28400 hybrid_clock.cc:648] HybridClock initialized: now 1790651381080271 us; error 28 us; skew 500 ppm
Sep 29 03:09:41 dist-test-slave-bdps krb5kdc[28387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651381, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:41.083287 28400 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:09:41.084237 28400 webserver.cc:460] Webserver started at http://127.0.0.1:39937/ using document root <none> and password file <none>
I20260929 03:09:41.084846 28400 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:41.084929 28400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:41.085139 28400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:41.087486 28400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "39ee61c2842c4fa4bf2062ea5ab4d1c1"
format_stamp: "Formatted at 2026-09-29 03:09:41 on dist-test-slave-bdps"
server_key: "f10119fe6fdf06ed74bad73b3ecfd96e"
server_key_iv: "1658b80051e6be64f1eb87af7ac96c4d"
server_key_version: "encryptionkey@0"
I20260929 03:09:41.088049 28400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "39ee61c2842c4fa4bf2062ea5ab4d1c1"
format_stamp: "Formatted at 2026-09-29 03:09:41 on dist-test-slave-bdps"
server_key: "f10119fe6fdf06ed74bad73b3ecfd96e"
server_key_iv: "1658b80051e6be64f1eb87af7ac96c4d"
server_key_version: "encryptionkey@0"
I20260929 03:09:41.092070 28400 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:09:41.094623 28415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:41.095904 28400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 03:09:41.096021 28400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "39ee61c2842c4fa4bf2062ea5ab4d1c1"
format_stamp: "Formatted at 2026-09-29 03:09:41 on dist-test-slave-bdps"
server_key: "f10119fe6fdf06ed74bad73b3ecfd96e"
server_key_iv: "1658b80051e6be64f1eb87af7ac96c4d"
server_key_version: "encryptionkey@0"
I20260929 03:09:41.096114 28400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:41.112013 28400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:41.115269 28400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:41.115471 28400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:41.123229 28400 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40037
I20260929 03:09:41.123292 28467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40037 every 8 connection(s)
I20260929 03:09:41.124341 28400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:41.127624 28468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:41.133076 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 28400
I20260929 03:09:41.133221 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:41.133528 8005 external_mini_cluster.cc:1428] Setting key db2b33d445f52cc75e90fd1114e5f344
I20260929 03:09:41.133802 28468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1: Bootstrap starting.
I20260929 03:09:41.136500 28468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:41.137423 28468 log.cc:824] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:41.139647 28468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1: No bootstrap required, opened a new log
I20260929 03:09:41.143150 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:41.134909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34393 (local address 127.0.0.1:40037)
0929 03:09:41.135326 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:41.135344 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:41.135385 (+ 41us) server_negotiation.cc:407] Connection header received
0929 03:09:41.136060 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:41.136078 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:41.136476 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:41.136816 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:41.137183 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:41.138283 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:41.139502 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:41.140235 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:41.140366 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:41.141302 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:41.141307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:41.141321 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:41.141792 (+ 471us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:41.141853 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:41.141877 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:41.141984 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:41.142181 (+ 197us) server_negotiation.cc:299] Negotiation successful
0929 03:09:41.142337 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":108,"threads_started":1}
I20260929 03:09:41.143741 28468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ee61c2842c4fa4bf2062ea5ab4d1c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40037 } }
I20260929 03:09:41.143972 28468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:41.144029 28468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39ee61c2842c4fa4bf2062ea5ab4d1c1, State: Initialized, Role: FOLLOWER
I20260929 03:09:41.144526 28468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ee61c2842c4fa4bf2062ea5ab4d1c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40037 } }
I20260929 03:09:41.144690 28468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:41.144759 28468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:41.144855 28468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:41.145915 28468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ee61c2842c4fa4bf2062ea5ab4d1c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40037 } }
I20260929 03:09:41.146330 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:41.146294 28468 leader_election.cc:302] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39ee61c2842c4fa4bf2062ea5ab4d1c1; no voters:
I20260929 03:09:41.146618 28468 leader_election.cc:288] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:41.146708 28473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:41.146937 28473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [term 1 LEADER]: Becoming Leader. State: Replica: 39ee61c2842c4fa4bf2062ea5ab4d1c1, State: Running, Role: LEADER
I20260929 03:09:41.147313 28473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ee61c2842c4fa4bf2062ea5ab4d1c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40037 } }
I20260929 03:09:41.148037 28468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:41.149060 28475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39ee61c2842c4fa4bf2062ea5ab4d1c1. Latest consensus state: current_term: 1 leader_uuid: "39ee61c2842c4fa4bf2062ea5ab4d1c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ee61c2842c4fa4bf2062ea5ab4d1c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40037 } } }
I20260929 03:09:41.149200 28475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:41.149518 28474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39ee61c2842c4fa4bf2062ea5ab4d1c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39ee61c2842c4fa4bf2062ea5ab4d1c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40037 } } }
I20260929 03:09:41.149596 28474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:41.150432 28487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:41.153862 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:41.148924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49573 (local address 127.0.0.1:40037)
0929 03:09:41.149025 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:41.149030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:41.149896 (+ 866us) server_negotiation.cc:407] Connection header received
0929 03:09:41.150073 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:41.150076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:41.150120 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:41.150172 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:41.150709 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:41.151348 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:41.152201 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:41.152440 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:41.152525 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:41.153375 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:41.153379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:41.153382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:41.153395 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:41.153422 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:41.153432 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:41.153497 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:41.153674 (+ 177us) server_negotiation.cc:299] Negotiation successful
0929 03:09:41.153710 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 03:09:41.153959 28487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:41.160555 28487 catalog_manager.cc:1347] Generated new cluster ID: c26e53ae81a242efb1297e8a77b8146b
I20260929 03:09:41.160698 28487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:41.170055 28487 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:41.171494 28487 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:41.182024 28487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39ee61c2842c4fa4bf2062ea5ab4d1c1: Generated new TSK 0
I20260929 03:09:41.183004 28487 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:41.191241 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 28400
W20260929 03:09:41.197700 28385 connection.cc:535] client connection to 127.0.0.1:40037 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40037): Connection reset by peer (error 104)
2026-09-29T03:09:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1326 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:41 dist-test-slave-bdps krb5kdc[28499](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:41 dist-test-slave-bdps krb5kdc[28499](info): set up 2 sockets
Sep 29 03:09:41 dist-test-slave-bdps krb5kdc[28499](info): commencing operation
krb5kdc: starting...
W20260929 03:09:42.150745 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.938s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:42 dist-test-slave-bdps krb5kdc[28499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:09:42.278515 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38185
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:42.403548 28512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:42.403903 28512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:42.403970 28512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:42.408130 28512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:42.408200 28512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:42.408226 28512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:42.408253 28512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:42.408281 28512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:42.413857 28512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38185
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:42.415261 28512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:42.416527 28512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:09:42.426297 28512 server_base.cc:1034] running on GCE node
W20260929 03:09:42.426257 28520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:42.426272 28518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:42.426257 28517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:42.427179 28512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:42.428455 28512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:42.429632 28512 hybrid_clock.cc:648] HybridClock initialized: now 1790651382429595 us; error 32 us; skew 500 ppm
Sep 29 03:09:42 dist-test-slave-bdps krb5kdc[28499](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651382, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:42.432493 28512 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:09:42.433485 28512 webserver.cc:460] Webserver started at http://127.0.0.1:37637/ using document root <none> and password file <none>
I20260929 03:09:42.434157 28512 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:42.434230 28512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:42.434463 28512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:42.436942 28512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "a396f80691cb4250a16f35f9fce8433e"
format_stamp: "Formatted at 2026-09-29 03:09:42 on dist-test-slave-bdps"
server_key: "c57e0184502c0aa96ef8c42cccacf632"
server_key_iv: "9d39e9f9eb75936a53b2922f56595011"
server_key_version: "encryptionkey@0"
I20260929 03:09:42.437554 28512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "a396f80691cb4250a16f35f9fce8433e"
format_stamp: "Formatted at 2026-09-29 03:09:42 on dist-test-slave-bdps"
server_key: "c57e0184502c0aa96ef8c42cccacf632"
server_key_iv: "9d39e9f9eb75936a53b2922f56595011"
server_key_version: "encryptionkey@0"
I20260929 03:09:42.441833 28512 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:09:42.444519 28527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:42.445915 28512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:42.446038 28512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "a396f80691cb4250a16f35f9fce8433e"
format_stamp: "Formatted at 2026-09-29 03:09:42 on dist-test-slave-bdps"
server_key: "c57e0184502c0aa96ef8c42cccacf632"
server_key_iv: "9d39e9f9eb75936a53b2922f56595011"
server_key_version: "encryptionkey@0"
I20260929 03:09:42.446137 28512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:42.451140 28512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:42.454234 28512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:42.454406 28512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:42.462113 28512 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38185
I20260929 03:09:42.462145 28579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38185 every 8 connection(s)
I20260929 03:09:42.463115 28512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:42.465706 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 28512
I20260929 03:09:42.465883 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:42.466233 8005 external_mini_cluster.cc:1428] Setting key ef542bae7a06208344d2ee06e686dc18
I20260929 03:09:42.466889 28580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:42.471995 28583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:42.467792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34205 (local address 127.0.0.1:38185)
0929 03:09:42.468256 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:42.468273 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:42.468312 (+ 39us) server_negotiation.cc:407] Connection header received
0929 03:09:42.469043 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:42.469063 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:42.469405 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:42.469797 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:42.470666 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:42.470671 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:42.470684 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:42.470694 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:42.470757 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:42.470778 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:42.470916 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:42.471000 (+ 84us) server_negotiation.cc:299] Negotiation successful
0929 03:09:42.471172 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":135,"threads_started":1}
I20260929 03:09:42.474229 28580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e: Bootstrap starting.
I20260929 03:09:42.475215 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:42.477181 28580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:42.478269 28580 log.cc:824] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:42.479276 28583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:42.477542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38143 (local address 127.0.0.1:38185)
0929 03:09:42.477717 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:42.477722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:42.477878 (+ 156us) server_negotiation.cc:407] Connection header received
0929 03:09:42.478018 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:42.478021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:42.478053 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:42.478118 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:42.478890 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:42.478894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:42.478897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:42.478908 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:42.478937 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:42.478950 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:42.479044 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:42.479099 (+ 55us) server_negotiation.cc:299] Negotiation successful
0929 03:09:42.479140 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:09:42.480787 28580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e: No bootstrap required, opened a new log
I20260929 03:09:42.484745 28580 raft_consensus.cc:348] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a396f80691cb4250a16f35f9fce8433e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } }
I20260929 03:09:42.485003 28580 raft_consensus.cc:374] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:42.485064 28580 raft_consensus.cc:729] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a396f80691cb4250a16f35f9fce8433e, State: Initialized, Role: FOLLOWER
I20260929 03:09:42.485540 28580 consensus_queue.cc:260] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a396f80691cb4250a16f35f9fce8433e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } }
I20260929 03:09:42.485728 28580 raft_consensus.cc:388] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:42.485795 28580 raft_consensus.cc:482] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:42.485899 28580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:42.486891 28580 raft_consensus.cc:504] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a396f80691cb4250a16f35f9fce8433e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } }
I20260929 03:09:42.487283 28580 leader_election.cc:302] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a396f80691cb4250a16f35f9fce8433e; no voters:
I20260929 03:09:42.487607 28580 leader_election.cc:288] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:42.487721 28590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:42.487963 28590 raft_consensus.cc:686] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [term 1 LEADER]: Becoming Leader. State: Replica: a396f80691cb4250a16f35f9fce8433e, State: Running, Role: LEADER
I20260929 03:09:42.488384 28590 consensus_queue.cc:237] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a396f80691cb4250a16f35f9fce8433e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } }
I20260929 03:09:42.490052 28580 sys_catalog.cc:564] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:42.490233 28591 sys_catalog.cc:455] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a396f80691cb4250a16f35f9fce8433e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a396f80691cb4250a16f35f9fce8433e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } } }
I20260929 03:09:42.490352 28591 sys_catalog.cc:458] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:42.490718 28592 sys_catalog.cc:455] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a396f80691cb4250a16f35f9fce8433e. Latest consensus state: current_term: 1 leader_uuid: "a396f80691cb4250a16f35f9fce8433e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a396f80691cb4250a16f35f9fce8433e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38185 } } }
I20260929 03:09:42.490798 28594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:42.490856 28592 sys_catalog.cc:458] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:42.494575 28594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:42.500773 28594 catalog_manager.cc:1347] Generated new cluster ID: 8b69525c08544b7aa2adf9d785e47ee3
I20260929 03:09:42.500846 28594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:42.517693 28594 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:42.518117 28594 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:42.527971 28594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a396f80691cb4250a16f35f9fce8433e: Generated new TSK 0
I20260929 03:09:42.528800 28594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:42.538122 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 28512
2026-09-29T03:09:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:42 dist-test-slave-bdps krb5kdc[28611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:42 dist-test-slave-bdps krb5kdc[28611](info): set up 2 sockets
Sep 29 03:09:42 dist-test-slave-bdps krb5kdc[28611](info): commencing operation
krb5kdc: starting...
W20260929 03:09:43.510031 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:43 dist-test-slave-bdps krb5kdc[28611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:09:43.638417 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46591
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:43.763761 28624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:43.764086 28624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:43.764147 28624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:43.768247 28624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:43.768311 28624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:43.768337 28624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:43.768359 28624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:43.768384 28624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:43.773690 28624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:43.774912 28624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:43.776367 28624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:09:43.785504 28624 server_base.cc:1034] running on GCE node
W20260929 03:09:43.785521 28630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:43.785509 28629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:43.785844 28632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:43.786404 28624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:43.787562 28624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:43.788715 28624 hybrid_clock.cc:648] HybridClock initialized: now 1790651383788700 us; error 31 us; skew 500 ppm
Sep 29 03:09:43 dist-test-slave-bdps krb5kdc[28611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651383, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:43.795295 28624 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:09:43.796378 28624 webserver.cc:460] Webserver started at http://127.0.0.1:38127/ using document root <none> and password file <none>
I20260929 03:09:43.797034 28624 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:43.797106 28624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:43.797344 28624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:43.799759 28624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "f8617175278c436ab3660827f63503f5"
format_stamp: "Formatted at 2026-09-29 03:09:43 on dist-test-slave-bdps"
server_key: "2b760ddb03f2115c477ccca262ef9eaf"
server_key_iv: "7c49c7cb7a0560f8b184211d445b49fc"
server_key_version: "encryptionkey@0"
I20260929 03:09:43.800354 28624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "f8617175278c436ab3660827f63503f5"
format_stamp: "Formatted at 2026-09-29 03:09:43 on dist-test-slave-bdps"
server_key: "2b760ddb03f2115c477ccca262ef9eaf"
server_key_iv: "7c49c7cb7a0560f8b184211d445b49fc"
server_key_version: "encryptionkey@0"
I20260929 03:09:43.804461 28624 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260929 03:09:43.807102 28639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:43.808445 28624 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 03:09:43.808562 28624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "f8617175278c436ab3660827f63503f5"
format_stamp: "Formatted at 2026-09-29 03:09:43 on dist-test-slave-bdps"
server_key: "2b760ddb03f2115c477ccca262ef9eaf"
server_key_iv: "7c49c7cb7a0560f8b184211d445b49fc"
server_key_version: "encryptionkey@0"
I20260929 03:09:43.808655 28624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:43.819653 28624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:43.822824 28624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:43.822991 28624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:43.830327 28624 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46591
I20260929 03:09:43.830344 28691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46591 every 8 connection(s)
I20260929 03:09:43.831312 28624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:43.834524 28692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:43.835216 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 28624
I20260929 03:09:43.835376 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:43.835700 8005 external_mini_cluster.cc:1428] Setting key 015c27f129d83b766d56e68848c5b485
I20260929 03:09:43.842314 28692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5: Bootstrap starting.
I20260929 03:09:43.845227 28692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5: Neither blocks nor log segments found. Creating new log.
Sep 29 03:09:43 dist-test-slave-bdps krb5kdc[28611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 03:09:43.846153 28692 log.cc:824] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:43.848735 28692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5: No bootstrap required, opened a new log
I20260929 03:09:43.851441 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:43.837293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51539 (local address 127.0.0.1:46591)
0929 03:09:43.837804 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:43.837827 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:43.837889 (+ 62us) server_negotiation.cc:407] Connection header received
0929 03:09:43.838791 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:43.838818 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:43.839322 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:43.839728 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:43.840440 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:43.842012 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:43.842946 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:43.843663 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:43.843842 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:43.846443 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:43.846450 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:43.846469 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:43.846502 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:43.848449 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:43.848877 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:43.848882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:43.848887 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:43.849004 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:43.849283 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:43.849286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:43.849287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:43.849779 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:43.849973 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:43.850260 (+ 287us) server_negotiation.cc:299] Negotiation successful
0929 03:09:43.850578 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":143,"threads_started":1}
I20260929 03:09:43.852687 28692 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8617175278c436ab3660827f63503f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46591 } }
I20260929 03:09:43.852933 28692 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:43.852991 28692 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8617175278c436ab3660827f63503f5, State: Initialized, Role: FOLLOWER
I20260929 03:09:43.853499 28692 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8617175278c436ab3660827f63503f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46591 } }
I20260929 03:09:43.853672 28692 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:43.853742 28692 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:43.853868 28692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:43.854588 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:43.854996 28692 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8617175278c436ab3660827f63503f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46591 } }
I20260929 03:09:43.855386 28692 leader_election.cc:302] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8617175278c436ab3660827f63503f5; no voters:
I20260929 03:09:43.855706 28692 leader_election.cc:288] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:43.855809 28697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:43.856055 28697 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [term 1 LEADER]: Becoming Leader. State: Replica: f8617175278c436ab3660827f63503f5, State: Running, Role: LEADER
I20260929 03:09:43.856426 28697 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8617175278c436ab3660827f63503f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46591 } }
I20260929 03:09:43.857193 28692 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:43.858290 28702 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8617175278c436ab3660827f63503f5. Latest consensus state: current_term: 1 leader_uuid: "f8617175278c436ab3660827f63503f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8617175278c436ab3660827f63503f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46591 } } }
I20260929 03:09:43.858415 28702 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:43.858729 28700 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8617175278c436ab3660827f63503f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8617175278c436ab3660827f63503f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46591 } } }
I20260929 03:09:43.858822 28700 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5 [sys.catalog]: This master's current role is: LEADER
W20260929 03:09:43.863353 28717 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:09:43.863446 28717 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:09:43.863507 28711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:43.864291 28711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:43.866432 28695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:43.856971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56435 (local address 127.0.0.1:46591)
0929 03:09:43.857088 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:43.857091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:43.857425 (+ 334us) server_negotiation.cc:407] Connection header received
0929 03:09:43.857538 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:43.857541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:43.857590 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:43.857691 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:43.858611 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:43.859458 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:43.860594 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:43.860833 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:43.860912 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:43.862669 (+ 1757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:43.862673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:43.862676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:43.862700 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:43.864471 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:43.864960 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:43.864964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:43.864966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:43.865069 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:43.865352 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:43.865356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:43.865357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:43.865601 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:43.865777 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:43.866134 (+ 357us) server_negotiation.cc:299] Negotiation successful
0929 03:09:43.866270 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:09:43.871390 28711 catalog_manager.cc:1347] Generated new cluster ID: 9d08bdbdb8064dc4b4e7fff1919aace7
I20260929 03:09:43.871457 28711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:43.880323 28711 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:43.881422 28711 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:43.892379 28711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8617175278c436ab3660827f63503f5: Generated new TSK 0
I20260929 03:09:43.893219 28711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:43.907946 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 28624
2026-09-29T03:09:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1364 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:43 dist-test-slave-bdps krb5kdc[28723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:43 dist-test-slave-bdps krb5kdc[28723](info): set up 2 sockets
Sep 29 03:09:43 dist-test-slave-bdps krb5kdc[28723](info): commencing operation
krb5kdc: starting...
W20260929 03:09:44.874123 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.945s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:44 dist-test-slave-bdps krb5kdc[28723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:09:45.000145 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35615
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:45.125648 28736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:45.125980 28736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:45.126045 28736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:45.130254 28736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:45.130307 28736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:45.130333 28736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:45.130355 28736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:45.130381 28736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:45.135680 28736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:45.136860 28736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:45.138060 28736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:09:45.147256 28736 server_base.cc:1034] running on GCE node
W20260929 03:09:45.147217 28744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:45.147219 28741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:45.147217 28742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:45.148128 28736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:45.149330 28736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:45.150508 28736 hybrid_clock.cc:648] HybridClock initialized: now 1790651385150482 us; error 37 us; skew 500 ppm
Sep 29 03:09:45 dist-test-slave-bdps krb5kdc[28723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651385, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:45.153451 28736 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:09:45.154495 28736 webserver.cc:460] Webserver started at http://127.0.0.1:33519/ using document root <none> and password file <none>
I20260929 03:09:45.155131 28736 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:45.155205 28736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:45.155437 28736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:45.157894 28736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "d3cde9484d6242c6826f675863dd5648"
format_stamp: "Formatted at 2026-09-29 03:09:45 on dist-test-slave-bdps"
server_key: "341ee6eda1df78ed63bd3e5af7ab87aa"
server_key_iv: "743e18384800ecefd3f0c38f0b9831e5"
server_key_version: "encryptionkey@0"
I20260929 03:09:45.158501 28736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "d3cde9484d6242c6826f675863dd5648"
format_stamp: "Formatted at 2026-09-29 03:09:45 on dist-test-slave-bdps"
server_key: "341ee6eda1df78ed63bd3e5af7ab87aa"
server_key_iv: "743e18384800ecefd3f0c38f0b9831e5"
server_key_version: "encryptionkey@0"
I20260929 03:09:45.162865 28736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 03:09:45.165560 28751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:45.167055 28736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:09:45.167179 28736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "d3cde9484d6242c6826f675863dd5648"
format_stamp: "Formatted at 2026-09-29 03:09:45 on dist-test-slave-bdps"
server_key: "341ee6eda1df78ed63bd3e5af7ab87aa"
server_key_iv: "743e18384800ecefd3f0c38f0b9831e5"
server_key_version: "encryptionkey@0"
I20260929 03:09:45.167284 28736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:45.181437 28736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:45.184881 28736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:45.185124 28736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:45.193450 28736 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35615
I20260929 03:09:45.193503 28803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35615 every 8 connection(s)
I20260929 03:09:45.194640 28736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:45.197154 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 28736
I20260929 03:09:45.197302 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:45.197651 8005 external_mini_cluster.cc:1428] Setting key 1e34ccc78bf552c749971470dd81ad80
I20260929 03:09:45.198475 28804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:45.206501 28804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648: Bootstrap starting.
I20260929 03:09:45.208715 28807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:45.199373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42219 (local address 127.0.0.1:35615)
0929 03:09:45.199858 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:45.199880 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:45.199938 (+ 58us) server_negotiation.cc:407] Connection header received
0929 03:09:45.200912 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:45.200940 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:45.201471 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:45.201937 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:45.202429 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:45.203995 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:45.204981 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:45.205770 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:45.205931 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:45.206837 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:45.206842 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:45.206857 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:45.207303 (+ 446us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:45.207364 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:45.207388 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:45.207479 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:45.207624 (+ 145us) server_negotiation.cc:299] Negotiation successful
0929 03:09:45.207878 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":142,"threads_started":1}
I20260929 03:09:45.209227 28804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:45.210170 28804 log.cc:824] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:45.211889 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:45.212611 28804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648: No bootstrap required, opened a new log
I20260929 03:09:45.216918 28804 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3cde9484d6242c6826f675863dd5648" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35615 } }
I20260929 03:09:45.217190 28804 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:45.217253 28804 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3cde9484d6242c6826f675863dd5648, State: Initialized, Role: FOLLOWER
I20260929 03:09:45.217833 28804 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3cde9484d6242c6826f675863dd5648" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35615 } }
I20260929 03:09:45.217991 28804 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:45.218060 28804 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:45.218165 28804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:45.218390 28807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:45.214481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48275 (local address 127.0.0.1:35615)
0929 03:09:45.214604 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:45.214609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:45.214853 (+ 244us) server_negotiation.cc:407] Connection header received
0929 03:09:45.214977 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:45.214979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:45.215039 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:45.215102 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:45.215374 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:45.215901 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:45.216756 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:45.216989 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:45.217078 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:45.217989 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:45.217994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:45.217997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:45.218010 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:45.218038 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:45.218051 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:45.218114 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:45.218193 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 03:09:45.218257 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 03:09:45.219430 28804 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3cde9484d6242c6826f675863dd5648" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35615 } }
I20260929 03:09:45.219794 28804 leader_election.cc:302] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3cde9484d6242c6826f675863dd5648; no voters:
I20260929 03:09:45.220062 28804 leader_election.cc:288] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:45.220160 28814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:45.220367 28814 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [term 1 LEADER]: Becoming Leader. State: Replica: d3cde9484d6242c6826f675863dd5648, State: Running, Role: LEADER
I20260929 03:09:45.220712 28814 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3cde9484d6242c6826f675863dd5648" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35615 } }
I20260929 03:09:45.221413 28804 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:45.222532 28816 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3cde9484d6242c6826f675863dd5648. Latest consensus state: current_term: 1 leader_uuid: "d3cde9484d6242c6826f675863dd5648" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3cde9484d6242c6826f675863dd5648" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35615 } } }
I20260929 03:09:45.222692 28816 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:45.222997 28815 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3cde9484d6242c6826f675863dd5648" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3cde9484d6242c6826f675863dd5648" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35615 } } }
I20260929 03:09:45.223158 28815 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:45.223085 28823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:45.227002 28823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:45.233501 28823 catalog_manager.cc:1347] Generated new cluster ID: 3dfe0d611e334ffd93efd931abe8af95
I20260929 03:09:45.233561 28823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:45.244814 28823 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:45.245863 28823 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:45.250913 28823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3cde9484d6242c6826f675863dd5648: Generated new TSK 0
I20260929 03:09:45.251633 28823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:45.265292 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 28736
2026-09-29T03:09:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:45 dist-test-slave-bdps krb5kdc[28835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:45 dist-test-slave-bdps krb5kdc[28835](info): set up 2 sockets
Sep 29 03:09:45 dist-test-slave-bdps krb5kdc[28835](info): commencing operation
krb5kdc: starting...
W20260929 03:09:46.217975 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:46 dist-test-slave-bdps krb5kdc[28835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:09:46.346360 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:46.471397 28848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:46.471788 28848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:46.471844 28848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:46.475983 28848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:46.476056 28848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:46.476076 28848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:46.476106 28848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:46.476135 28848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:46.481884 28848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:46.483186 28848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:46.484385 28848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:46.493800 28854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:46.493810 28853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:46.493877 28848 server_base.cc:1034] running on GCE node
W20260929 03:09:46.493829 28856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:46.494609 28848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:46.495844 28848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:46.497005 28848 hybrid_clock.cc:648] HybridClock initialized: now 1790651386496993 us; error 28 us; skew 500 ppm
Sep 29 03:09:46 dist-test-slave-bdps krb5kdc[28835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651386, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:46.499828 28848 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:09:46.500715 28848 webserver.cc:460] Webserver started at http://127.0.0.1:36219/ using document root <none> and password file <none>
I20260929 03:09:46.501299 28848 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:46.501368 28848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:46.501580 28848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:46.503798 28848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "fcd47c72e4c44884b476195ef1a0b676"
format_stamp: "Formatted at 2026-09-29 03:09:46 on dist-test-slave-bdps"
server_key: "0ba57e66709888d0b8bcad7dd84d3e03"
server_key_iv: "36f5088e2c462ea33f617fac4a31e584"
server_key_version: "encryptionkey@0"
I20260929 03:09:46.504376 28848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "fcd47c72e4c44884b476195ef1a0b676"
format_stamp: "Formatted at 2026-09-29 03:09:46 on dist-test-slave-bdps"
server_key: "0ba57e66709888d0b8bcad7dd84d3e03"
server_key_iv: "36f5088e2c462ea33f617fac4a31e584"
server_key_version: "encryptionkey@0"
I20260929 03:09:46.508252 28848 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260929 03:09:46.510725 28863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:46.511981 28848 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 03:09:46.512096 28848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "fcd47c72e4c44884b476195ef1a0b676"
format_stamp: "Formatted at 2026-09-29 03:09:46 on dist-test-slave-bdps"
server_key: "0ba57e66709888d0b8bcad7dd84d3e03"
server_key_iv: "36f5088e2c462ea33f617fac4a31e584"
server_key_version: "encryptionkey@0"
I20260929 03:09:46.512194 28848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:46.516884 28848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:46.520046 28848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:46.520249 28848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:46.527704 28848 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40017
I20260929 03:09:46.527714 28915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40017 every 8 connection(s)
I20260929 03:09:46.528702 28848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:46.531838 28916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:46.532624 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 28848
I20260929 03:09:46.532783 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:46.533090 8005 external_mini_cluster.cc:1428] Setting key 218f544c5ab2a2fa92968757f2671429
I20260929 03:09:46.538919 28919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:46.534576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42917 (local address 127.0.0.1:40017)
0929 03:09:46.535040 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:46.535060 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:46.535106 (+ 46us) server_negotiation.cc:407] Connection header received
0929 03:09:46.535976 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:46.536000 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:46.536394 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:46.536788 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:46.537449 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:46.537454 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:46.537471 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:46.537481 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:46.537540 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:46.537567 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:46.537770 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:46.537848 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 03:09:46.538007 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":123,"threads_started":1}
I20260929 03:09:46.539029 28916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676: Bootstrap starting.
I20260929 03:09:46.541810 28916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:46.541961 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:46.542606 28916 log.cc:824] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:46.544804 28916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676: No bootstrap required, opened a new log
I20260929 03:09:46.546003 28919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:46.544233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52967 (local address 127.0.0.1:40017)
0929 03:09:46.544332 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:46.544336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:46.544516 (+ 180us) server_negotiation.cc:407] Connection header received
0929 03:09:46.544623 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:46.544627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:46.544662 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:46.544735 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:46.545518 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:46.545522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:46.545524 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:46.545537 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:46.545581 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:46.545593 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:46.545723 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:46.545842 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 03:09:46.545884 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 03:09:46.549034 28916 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd47c72e4c44884b476195ef1a0b676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40017 } }
I20260929 03:09:46.549266 28916 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:46.549330 28916 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcd47c72e4c44884b476195ef1a0b676, State: Initialized, Role: FOLLOWER
I20260929 03:09:46.549839 28916 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd47c72e4c44884b476195ef1a0b676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40017 } }
I20260929 03:09:46.550010 28916 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:46.550069 28916 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:46.550146 28916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:46.550995 28916 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd47c72e4c44884b476195ef1a0b676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40017 } }
I20260929 03:09:46.551330 28916 leader_election.cc:302] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcd47c72e4c44884b476195ef1a0b676; no voters:
I20260929 03:09:46.551590 28916 leader_election.cc:288] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:46.551703 28926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:46.551937 28926 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [term 1 LEADER]: Becoming Leader. State: Replica: fcd47c72e4c44884b476195ef1a0b676, State: Running, Role: LEADER
I20260929 03:09:46.552268 28926 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd47c72e4c44884b476195ef1a0b676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40017 } }
I20260929 03:09:46.552815 28916 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:46.553985 28928 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcd47c72e4c44884b476195ef1a0b676. Latest consensus state: current_term: 1 leader_uuid: "fcd47c72e4c44884b476195ef1a0b676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd47c72e4c44884b476195ef1a0b676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40017 } } }
I20260929 03:09:46.554121 28928 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:46.554522 28935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:46.554498 28927 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcd47c72e4c44884b476195ef1a0b676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcd47c72e4c44884b476195ef1a0b676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40017 } } }
I20260929 03:09:46.554714 28927 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:46.557957 28935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:46.563995 28935 catalog_manager.cc:1347] Generated new cluster ID: 23900cdbe3a2422cac9289e6b45af647
I20260929 03:09:46.564085 28935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:46.578302 28935 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:46.578804 28935 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:46.593169 28935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcd47c72e4c44884b476195ef1a0b676: Generated new TSK 0
I20260929 03:09:46.594214 28935 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:46.606383 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 28848
2026-09-29T03:09:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:46 dist-test-slave-bdps krb5kdc[28947](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:46 dist-test-slave-bdps krb5kdc[28947](info): set up 2 sockets
Sep 29 03:09:46 dist-test-slave-bdps krb5kdc[28947](info): commencing operation
krb5kdc: starting...
W20260929 03:09:47.554184 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:47 dist-test-slave-bdps krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:09:47.679221 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:47.803535 28960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:47.803843 28960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:47.803885 28960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:47.807946 28960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:47.808002 28960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:47.808020 28960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:47.808038 28960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:47.808056 28960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:47.813199 28960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:47.814312 28960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:47.815419 28960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:47.824494 28966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:47.824517 28965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:47.824553 28968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:47.824771 28960 server_base.cc:1034] running on GCE node
I20260929 03:09:47.825378 28960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:47.826647 28960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:47.827823 28960 hybrid_clock.cc:648] HybridClock initialized: now 1790651387827810 us; error 30 us; skew 500 ppm
Sep 29 03:09:47 dist-test-slave-bdps krb5kdc[28947](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651387, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:47.830675 28960 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:09:47.831606 28960 webserver.cc:460] Webserver started at http://127.0.0.1:46265/ using document root <none> and password file <none>
I20260929 03:09:47.832211 28960 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:47.832288 28960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:47.832515 28960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:47.834892 28960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "375b579919ef42a69588106bac0056c2"
format_stamp: "Formatted at 2026-09-29 03:09:47 on dist-test-slave-bdps"
server_key: "551e50baa1708a8c6ee7dbaa0b67a396"
server_key_iv: "8e13a80a53773b6b26d1314386ce1ba3"
server_key_version: "encryptionkey@0"
I20260929 03:09:47.835533 28960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "375b579919ef42a69588106bac0056c2"
format_stamp: "Formatted at 2026-09-29 03:09:47 on dist-test-slave-bdps"
server_key: "551e50baa1708a8c6ee7dbaa0b67a396"
server_key_iv: "8e13a80a53773b6b26d1314386ce1ba3"
server_key_version: "encryptionkey@0"
I20260929 03:09:47.840013 28960 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:09:47.842826 28975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:47.844203 28960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:09:47.844340 28960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "375b579919ef42a69588106bac0056c2"
format_stamp: "Formatted at 2026-09-29 03:09:47 on dist-test-slave-bdps"
server_key: "551e50baa1708a8c6ee7dbaa0b67a396"
server_key_iv: "8e13a80a53773b6b26d1314386ce1ba3"
server_key_version: "encryptionkey@0"
I20260929 03:09:47.844455 28960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:47.858511 28960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:47.861907 28960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:47.862128 28960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:47.870322 28960 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40717
I20260929 03:09:47.870358 29027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40717 every 8 connection(s)
I20260929 03:09:47.871481 28960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:47.874913 29028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:47.876644 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 28960
I20260929 03:09:47.876780 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:47.877103 8005 external_mini_cluster.cc:1428] Setting key 7f347a908b5aa0a644cdf180214d89bc
I20260929 03:09:47.882391 29028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2: Bootstrap starting.
I20260929 03:09:47.885185 29028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:47.886076 29028 log.cc:824] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2: Log is configured to *not* fsync() on all Append() calls
Sep 29 03:09:47 dist-test-slave-bdps krb5kdc[28947](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 03:09:47.888458 29028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2: No bootstrap required, opened a new log
I20260929 03:09:47.892421 29031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:47.878852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.117:49085 (local address 127.0.0.1:40717)
0929 03:09:47.879346 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:47.879365 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:47.879407 (+ 42us) server_negotiation.cc:407] Connection header received
0929 03:09:47.880122 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:47.880143 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:47.880557 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:47.880922 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:47.881833 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:47.883000 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:47.883924 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:47.884437 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:47.884592 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:47.887492 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:47.887497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:47.887512 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:47.887540 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:47.889221 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:47.889775 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:47.889780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:47.889785 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:47.889920 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:47.890261 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:47.890263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:47.890264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:47.890724 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:47.890947 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:47.891282 (+ 335us) server_negotiation.cc:299] Negotiation successful
0929 03:09:47.891523 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":153,"threads_started":1}
I20260929 03:09:47.892436 29028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375b579919ef42a69588106bac0056c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40717 } }
I20260929 03:09:47.892788 29028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:47.892848 29028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 375b579919ef42a69588106bac0056c2, State: Initialized, Role: FOLLOWER
I20260929 03:09:47.893361 29028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375b579919ef42a69588106bac0056c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40717 } }
I20260929 03:09:47.893496 29028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:47.893561 29028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:47.893692 29028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:47.894927 29028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375b579919ef42a69588106bac0056c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40717 } }
I20260929 03:09:47.895325 29028 leader_election.cc:302] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 375b579919ef42a69588106bac0056c2; no voters:
I20260929 03:09:47.895695 29028 leader_election.cc:288] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:47.895782 29033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:47.895982 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:47.896034 29033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [term 1 LEADER]: Becoming Leader. State: Replica: 375b579919ef42a69588106bac0056c2, State: Running, Role: LEADER
I20260929 03:09:47.896384 29033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375b579919ef42a69588106bac0056c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40717 } }
I20260929 03:09:47.897251 29028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:47.897542 29033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 375b579919ef42a69588106bac0056c2. Latest consensus state: current_term: 1 leader_uuid: "375b579919ef42a69588106bac0056c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375b579919ef42a69588106bac0056c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40717 } } }
I20260929 03:09:47.897684 29033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:47.898038 29034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "375b579919ef42a69588106bac0056c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "375b579919ef42a69588106bac0056c2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40717 } } }
I20260929 03:09:47.898118 29034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:47.898176 29041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:47.901829 29041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:47.908154 29031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:47.898728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.117:40291 (local address 127.0.0.1:40717)
0929 03:09:47.898847 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:47.898852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:47.899246 (+ 394us) server_negotiation.cc:407] Connection header received
0929 03:09:47.899444 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:47.899449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:47.899503 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:47.899561 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:47.900549 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:47.901325 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:47.902508 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:47.902843 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:47.902919 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:47.904864 (+ 1945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:47.904867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:47.904869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:47.904889 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:47.906268 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:47.906816 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:47.906819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:47.906821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:47.906907 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:47.907252 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:47.907255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:47.907258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:47.907466 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:47.907602 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:47.907905 (+ 303us) server_negotiation.cc:299] Negotiation successful
0929 03:09:47.907998 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 03:09:47.911062 29041 catalog_manager.cc:1347] Generated new cluster ID: f18a3924c0764a3a8f342c526a22c5ea
I20260929 03:09:47.911118 29041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:47.933990 29041 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:47.935117 29041 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:47.944000 29041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 375b579919ef42a69588106bac0056c2: Generated new TSK 0
I20260929 03:09:47.944792 29041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:47.954152 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 28960
W20260929 03:09:47.960213 28945 connection.cc:535] client connection to 127.0.0.1:40717 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40717): Connection reset by peer (error 104)
2026-09-29T03:09:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:48 dist-test-slave-bdps krb5kdc[29059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:48 dist-test-slave-bdps krb5kdc[29059](info): set up 2 sockets
Sep 29 03:09:48 dist-test-slave-bdps krb5kdc[29059](info): commencing operation
krb5kdc: starting...
W20260929 03:09:48.910010 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:48 dist-test-slave-bdps krb5kdc[29059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:09:49.037949 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42069
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:49.160123 29072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:49.160506 29072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:49.160573 29072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:49.164762 29072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:49.164850 29072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:49.164878 29072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:49.164899 29072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:49.164917 29072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:49.170686 29072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:49.172048 29072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:49.173394 29072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:49.182879 29077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:49.182862 29078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:49.183123 29072 server_base.cc:1034] running on GCE node
W20260929 03:09:49.183148 29080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:49.183732 29072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:49.184963 29072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:49.186126 29072 hybrid_clock.cc:648] HybridClock initialized: now 1790651389186112 us; error 27 us; skew 500 ppm
Sep 29 03:09:49 dist-test-slave-bdps krb5kdc[29059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651389, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:49.194345 29072 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:09:49.195340 29072 webserver.cc:460] Webserver started at http://127.0.0.1:39793/ using document root <none> and password file <none>
I20260929 03:09:49.195961 29072 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:49.196022 29072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:49.196233 29072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:49.198704 29072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "7c19186b0f0e42248b1ee1a3e51c2f89"
format_stamp: "Formatted at 2026-09-29 03:09:49 on dist-test-slave-bdps"
server_key: "b8f2aab8f08ad32e411319bbbf0c3323"
server_key_iv: "347421d41b7b2da7f529384802ca7031"
server_key_version: "encryptionkey@0"
I20260929 03:09:49.199302 29072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "7c19186b0f0e42248b1ee1a3e51c2f89"
format_stamp: "Formatted at 2026-09-29 03:09:49 on dist-test-slave-bdps"
server_key: "b8f2aab8f08ad32e411319bbbf0c3323"
server_key_iv: "347421d41b7b2da7f529384802ca7031"
server_key_version: "encryptionkey@0"
I20260929 03:09:49.203927 29072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 03:09:49.207041 29087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:49.208437 29072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260929 03:09:49.208545 29072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "7c19186b0f0e42248b1ee1a3e51c2f89"
format_stamp: "Formatted at 2026-09-29 03:09:49 on dist-test-slave-bdps"
server_key: "b8f2aab8f08ad32e411319bbbf0c3323"
server_key_iv: "347421d41b7b2da7f529384802ca7031"
server_key_version: "encryptionkey@0"
I20260929 03:09:49.208633 29072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:49.219434 29072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:49.222226 29072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:49.222375 29072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:49.229705 29072 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42069
I20260929 03:09:49.229753 29139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42069 every 8 connection(s)
I20260929 03:09:49.230705 29072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:49.233821 29140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:49.233851 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 29072
I20260929 03:09:49.233974 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:49.234292 8005 external_mini_cluster.cc:1428] Setting key 92d88092daa0f9046b39339195261909
I20260929 03:09:49.241643 29140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89: Bootstrap starting.
I20260929 03:09:49.244742 29140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:49.245087 29143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:49.235719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.117:60159 (local address 127.0.0.1:42069)
0929 03:09:49.236160 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:49.236179 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:49.236226 (+ 47us) server_negotiation.cc:407] Connection header received
0929 03:09:49.237292 (+ 1066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:49.237320 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:49.237833 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:49.238260 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:49.238687 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:49.240196 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:49.241135 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:49.241881 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:49.242038 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:49.242945 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:49.242953 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:49.242975 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:49.243534 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:49.243603 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:49.243632 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:49.243762 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:49.243874 (+ 112us) server_negotiation.cc:299] Negotiation successful
0929 03:09:49.244019 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":130,"threads_started":1}
I20260929 03:09:49.245711 29140 log.cc:824] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:49.248036 29140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89: No bootstrap required, opened a new log
I20260929 03:09:49.248358 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:49.252241 29140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c19186b0f0e42248b1ee1a3e51c2f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } }
I20260929 03:09:49.252552 29140 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:49.252609 29140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c19186b0f0e42248b1ee1a3e51c2f89, State: Initialized, Role: FOLLOWER
I20260929 03:09:49.253111 29140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c19186b0f0e42248b1ee1a3e51c2f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } }
I20260929 03:09:49.253264 29140 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:49.253371 29140 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:49.253477 29140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:49.254489 29140 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c19186b0f0e42248b1ee1a3e51c2f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } }
I20260929 03:09:49.254537 29143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:49.250587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.117:34791 (local address 127.0.0.1:42069)
0929 03:09:49.250696 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:49.250700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:49.250861 (+ 161us) server_negotiation.cc:407] Connection header received
0929 03:09:49.250987 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:49.250989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:49.251028 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:49.251088 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:49.251369 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:49.251873 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:49.252948 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:49.253183 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:49.253284 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:49.254125 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:49.254128 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:49.254130 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:49.254142 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:49.254166 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:49.254177 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:49.254257 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:49.254371 (+ 114us) server_negotiation.cc:299] Negotiation successful
0929 03:09:49.254405 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 03:09:49.254899 29140 leader_election.cc:302] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c19186b0f0e42248b1ee1a3e51c2f89; no voters:
I20260929 03:09:49.255241 29140 leader_election.cc:288] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:49.255318 29150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:49.255545 29150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [term 1 LEADER]: Becoming Leader. State: Replica: 7c19186b0f0e42248b1ee1a3e51c2f89, State: Running, Role: LEADER
I20260929 03:09:49.255939 29150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c19186b0f0e42248b1ee1a3e51c2f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } }
I20260929 03:09:49.256543 29140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:49.257746 29152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c19186b0f0e42248b1ee1a3e51c2f89. Latest consensus state: current_term: 1 leader_uuid: "7c19186b0f0e42248b1ee1a3e51c2f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c19186b0f0e42248b1ee1a3e51c2f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } } }
I20260929 03:09:49.257791 29151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c19186b0f0e42248b1ee1a3e51c2f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c19186b0f0e42248b1ee1a3e51c2f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42069 } } }
I20260929 03:09:49.257881 29151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:49.257866 29152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:49.258342 29159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:49.261870 29159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:49.268321 29159 catalog_manager.cc:1347] Generated new cluster ID: 52388fe825624b6690a6f6fa02f8ccab
I20260929 03:09:49.268394 29159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:49.273865 29159 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:49.275326 29159 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:49.287393 29159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c19186b0f0e42248b1ee1a3e51c2f89: Generated new TSK 0
I20260929 03:09:49.288401 29159 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:49.295291 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 29072
W20260929 03:09:49.301364 29057 connection.cc:535] client connection to 127.0.0.1:42069 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42069): Connection reset by peer (error 104)
2026-09-29T03:09:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:49 dist-test-slave-bdps krb5kdc[29171](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:49 dist-test-slave-bdps krb5kdc[29171](info): set up 2 sockets
Sep 29 03:09:49 dist-test-slave-bdps krb5kdc[29171](info): commencing operation
krb5kdc: starting...
W20260929 03:09:50.253988 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:50 dist-test-slave-bdps krb5kdc[29171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:09:50.376575 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35489
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:50.503069 29184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:50.503410 29184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:50.503475 29184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:50.507560 29184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:50.507630 29184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:50.507678 29184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:50.507701 29184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:50.507727 29184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:50.513167 29184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:50.514462 29184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:50.515765 29184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 03:09:50.525837 29184 server_base.cc:1034] running on GCE node
W20260929 03:09:50.525787 29190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:50.525797 29189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:50.525800 29192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:50.526701 29184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:50.528079 29184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:50.529238 29184 hybrid_clock.cc:648] HybridClock initialized: now 1790651390529219 us; error 34 us; skew 500 ppm
Sep 29 03:09:50 dist-test-slave-bdps krb5kdc[29171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651390, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:50.532181 29184 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:09:50.533223 29184 webserver.cc:460] Webserver started at http://127.0.0.1:33169/ using document root <none> and password file <none>
I20260929 03:09:50.533931 29184 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:50.534003 29184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:50.534241 29184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:50.536615 29184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "4e9ec9b84e2844d781133d1bb59af0c1"
format_stamp: "Formatted at 2026-09-29 03:09:50 on dist-test-slave-bdps"
server_key: "f1aec577101489b8265a8d278d4a9f4c"
server_key_iv: "f46118c224f04838bb0b03638a51a1a0"
server_key_version: "encryptionkey@0"
I20260929 03:09:50.537233 29184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "4e9ec9b84e2844d781133d1bb59af0c1"
format_stamp: "Formatted at 2026-09-29 03:09:50 on dist-test-slave-bdps"
server_key: "f1aec577101489b8265a8d278d4a9f4c"
server_key_iv: "f46118c224f04838bb0b03638a51a1a0"
server_key_version: "encryptionkey@0"
I20260929 03:09:50.541332 29184 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260929 03:09:50.543943 29199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:50.545291 29184 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260929 03:09:50.545419 29184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "4e9ec9b84e2844d781133d1bb59af0c1"
format_stamp: "Formatted at 2026-09-29 03:09:50 on dist-test-slave-bdps"
server_key: "f1aec577101489b8265a8d278d4a9f4c"
server_key_iv: "f46118c224f04838bb0b03638a51a1a0"
server_key_version: "encryptionkey@0"
I20260929 03:09:50.545516 29184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:50.550411 29184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:50.553217 29184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:50.553383 29184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:50.561409 29184 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35489
I20260929 03:09:50.561443 29251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35489 every 8 connection(s)
I20260929 03:09:50.562561 29184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:50.563213 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 29184
I20260929 03:09:50.563351 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:50.563678 8005 external_mini_cluster.cc:1428] Setting key db84ef5d3a3ea3920c70a70da760b566
I20260929 03:09:50.566198 29252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:50.569566 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:50.565313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.117:56091 (local address 127.0.0.1:35489)
0929 03:09:50.565788 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:50.565806 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:50.565848 (+ 42us) server_negotiation.cc:407] Connection header received
0929 03:09:50.566598 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:50.566618 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:50.566972 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:50.567302 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:50.568216 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:50.568221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:50.568237 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:50.568247 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:50.568306 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:50.568326 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:50.568441 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:50.568547 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 03:09:50.568714 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":129,"threads_started":1}
I20260929 03:09:50.572546 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:50.573110 29252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1: Bootstrap starting.
I20260929 03:09:50.575757 29252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:50.576421 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:50.574780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.117:50867 (local address 127.0.0.1:35489)
0929 03:09:50.574890 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:50.574894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:50.575107 (+ 213us) server_negotiation.cc:407] Connection header received
0929 03:09:50.575218 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:50.575221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:50.575260 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:50.575323 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:50.576046 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:50.576049 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:50.576052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:50.576063 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:50.576092 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:50.576102 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:50.576172 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:50.576262 (+ 90us) server_negotiation.cc:299] Negotiation successful
0929 03:09:50.576299 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 03:09:50.576700 29252 log.cc:824] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:50.578927 29252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1: No bootstrap required, opened a new log
I20260929 03:09:50.583132 29252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9ec9b84e2844d781133d1bb59af0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35489 } }
I20260929 03:09:50.583441 29252 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:50.583509 29252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e9ec9b84e2844d781133d1bb59af0c1, State: Initialized, Role: FOLLOWER
I20260929 03:09:50.584028 29252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9ec9b84e2844d781133d1bb59af0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35489 } }
I20260929 03:09:50.584168 29252 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:50.584249 29252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:50.584353 29252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:50.585426 29252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9ec9b84e2844d781133d1bb59af0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35489 } }
I20260929 03:09:50.585835 29252 leader_election.cc:302] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e9ec9b84e2844d781133d1bb59af0c1; no voters:
I20260929 03:09:50.586251 29252 leader_election.cc:288] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:50.586478 29262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:50.586769 29262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [term 1 LEADER]: Becoming Leader. State: Replica: 4e9ec9b84e2844d781133d1bb59af0c1, State: Running, Role: LEADER
I20260929 03:09:50.587085 29262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9ec9b84e2844d781133d1bb59af0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35489 } }
I20260929 03:09:50.587625 29252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:50.588950 29264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e9ec9b84e2844d781133d1bb59af0c1. Latest consensus state: current_term: 1 leader_uuid: "4e9ec9b84e2844d781133d1bb59af0c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9ec9b84e2844d781133d1bb59af0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35489 } } }
I20260929 03:09:50.588977 29263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e9ec9b84e2844d781133d1bb59af0c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9ec9b84e2844d781133d1bb59af0c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35489 } } }
I20260929 03:09:50.589075 29263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:50.589074 29264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:50.589489 29272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:50.592700 29272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:50.598851 29272 catalog_manager.cc:1347] Generated new cluster ID: 469594de8b7a41d29fe0f442c3b2b193
I20260929 03:09:50.598922 29272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:50.635361 29272 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:50.635773 29272 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:50.651805 29272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e9ec9b84e2844d781133d1bb59af0c1: Generated new TSK 0
I20260929 03:09:50.652873 29272 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:50.670857 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 29184
2026-09-29T03:09:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1375 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:50 dist-test-slave-bdps krb5kdc[29283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:50 dist-test-slave-bdps krb5kdc[29283](info): set up 2 sockets
Sep 29 03:09:50 dist-test-slave-bdps krb5kdc[29283](info): commencing operation
krb5kdc: starting...
W20260929 03:09:51.622107 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:51 dist-test-slave-bdps krb5kdc[29283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:09:51.751942 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:51.872071 29296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:51.872390 29296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:51.872453 29296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:51.876477 29296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:51.876555 29296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:51.876582 29296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:51.876603 29296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:51.876622 29296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:51.882380 29296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33695
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:51.883919 29296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:51.885298 29296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:51.895629 29304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:51.895629 29302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:51.895709 29301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:51.896005 29296 server_base.cc:1034] running on GCE node
I20260929 03:09:51.896694 29296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:51.898141 29296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:51.899312 29296 hybrid_clock.cc:648] HybridClock initialized: now 1790651391899291 us; error 36 us; skew 500 ppm
Sep 29 03:09:51 dist-test-slave-bdps krb5kdc[29283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651391, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:51.902380 29296 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:09:51.903385 29296 webserver.cc:460] Webserver started at http://127.0.0.1:45073/ using document root <none> and password file <none>
I20260929 03:09:51.904019 29296 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:51.904083 29296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:51.904320 29296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:51.906747 29296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "72e4a18d40394dc9a4c6f052df28b750"
format_stamp: "Formatted at 2026-09-29 03:09:51 on dist-test-slave-bdps"
server_key: "817928420262a9d2efed32be913e9169"
server_key_iv: "e67b163a347f5c36842c1a5e2e70ec1d"
server_key_version: "encryptionkey@0"
I20260929 03:09:51.907357 29296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "72e4a18d40394dc9a4c6f052df28b750"
format_stamp: "Formatted at 2026-09-29 03:09:51 on dist-test-slave-bdps"
server_key: "817928420262a9d2efed32be913e9169"
server_key_iv: "e67b163a347f5c36842c1a5e2e70ec1d"
server_key_version: "encryptionkey@0"
I20260929 03:09:51.911502 29296 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:09:51.914062 29311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:51.915401 29296 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 03:09:51.915505 29296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "72e4a18d40394dc9a4c6f052df28b750"
format_stamp: "Formatted at 2026-09-29 03:09:51 on dist-test-slave-bdps"
server_key: "817928420262a9d2efed32be913e9169"
server_key_iv: "e67b163a347f5c36842c1a5e2e70ec1d"
server_key_version: "encryptionkey@0"
I20260929 03:09:51.915592 29296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:51.933144 29296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:51.936668 29296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:51.936846 29296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:51.944394 29296 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33695
I20260929 03:09:51.944413 29363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33695 every 8 connection(s)
I20260929 03:09:51.945470 29296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:51.948244 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 29296
I20260929 03:09:51.948374 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:51.948674 29364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:51.948689 8005 external_mini_cluster.cc:1428] Setting key ab530268284883f8c5c71894bb14bb43
I20260929 03:09:51.956523 29364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750: Bootstrap starting.
Sep 29 03:09:51 dist-test-slave-bdps krb5kdc[29283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 03:09:51.959462 29364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:51.960294 29364 log.cc:824] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:51.962788 29364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750: No bootstrap required, opened a new log
I20260929 03:09:51.964285 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:51.950182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.117:41875 (local address 127.0.0.1:33695)
0929 03:09:51.950620 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:51.950640 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:51.950686 (+ 46us) server_negotiation.cc:407] Connection header received
0929 03:09:51.951602 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:51.951628 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:51.952116 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:51.952516 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:51.953196 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:51.954724 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:51.955611 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:51.956321 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:51.956497 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:51.959068 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:51.959075 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:51.959095 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:51.959131 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:51.960871 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:51.961418 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:51.961424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:51.961430 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:51.961574 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:51.961909 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:51.961912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:51.961913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:51.962451 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:51.962675 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:51.962961 (+ 286us) server_negotiation.cc:299] Negotiation successful
0929 03:09:51.963219 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":124,"threads_started":1}
I20260929 03:09:51.967078 29364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e4a18d40394dc9a4c6f052df28b750" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } }
I20260929 03:09:51.967335 29364 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:51.967417 29364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72e4a18d40394dc9a4c6f052df28b750, State: Initialized, Role: FOLLOWER
I20260929 03:09:51.967489 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:51.967950 29364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e4a18d40394dc9a4c6f052df28b750" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } }
I20260929 03:09:51.968116 29364 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:51.968181 29364 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:51.968291 29364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:51.969650 29364 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e4a18d40394dc9a4c6f052df28b750" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } }
I20260929 03:09:51.970063 29364 leader_election.cc:302] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72e4a18d40394dc9a4c6f052df28b750; no voters:
I20260929 03:09:51.970451 29364 leader_election.cc:288] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:51.970530 29374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:51.970780 29374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [term 1 LEADER]: Becoming Leader. State: Replica: 72e4a18d40394dc9a4c6f052df28b750, State: Running, Role: LEADER
I20260929 03:09:51.971197 29374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e4a18d40394dc9a4c6f052df28b750" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } }
I20260929 03:09:51.971864 29364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:51.972962 29375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72e4a18d40394dc9a4c6f052df28b750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e4a18d40394dc9a4c6f052df28b750" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } } }
I20260929 03:09:51.973088 29375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:51.973445 29376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72e4a18d40394dc9a4c6f052df28b750. Latest consensus state: current_term: 1 leader_uuid: "72e4a18d40394dc9a4c6f052df28b750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72e4a18d40394dc9a4c6f052df28b750" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33695 } } }
I20260929 03:09:51.973574 29376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750 [sys.catalog]: This master's current role is: LEADER
W20260929 03:09:51.978484 29389 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:09:51.978561 29389 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:09:51.978617 29383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:51.979426 29367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:51.970101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.117:44035 (local address 127.0.0.1:33695)
0929 03:09:51.970208 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:51.970212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:51.970489 (+ 277us) server_negotiation.cc:407] Connection header received
0929 03:09:51.970638 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:51.970641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:51.970696 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:51.970797 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:51.971788 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:51.972571 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:51.973839 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:51.974162 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:51.974295 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:51.975761 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:51.975765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:51.975768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:51.975792 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:51.977400 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:51.977965 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:51.977968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:51.977970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:51.978058 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:51.978396 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:51.978399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:51.978401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:51.978629 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:51.978836 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:51.979152 (+ 316us) server_negotiation.cc:299] Negotiation successful
0929 03:09:51.979270 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 03:09:51.979707 29383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:51.985526 29383 catalog_manager.cc:1347] Generated new cluster ID: 7c5b29f5281d49298ab111046666e934
I20260929 03:09:51.985585 29383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:52.004434 29383 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:52.005604 29383 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:52.015900 29383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72e4a18d40394dc9a4c6f052df28b750: Generated new TSK 0
I20260929 03:09:52.016993 29383 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:52.022147 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 29296
W20260929 03:09:52.028170 29281 connection.cc:535] client connection to 127.0.0.1:33695 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33695): Connection reset by peer (error 104)
2026-09-29T03:09:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:52 dist-test-slave-bdps krb5kdc[29395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:52 dist-test-slave-bdps krb5kdc[29395](info): set up 2 sockets
Sep 29 03:09:52 dist-test-slave-bdps krb5kdc[29395](info): commencing operation
krb5kdc: starting...
W20260929 03:09:52.978058 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:53 dist-test-slave-bdps krb5kdc[29395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:09:53.105777 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45307
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:53.232263 29408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:53.232569 29408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:53.232612 29408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:53.236691 29408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:53.236755 29408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:53.236774 29408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:53.236790 29408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:53.236804 29408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:53.242420 29408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45307
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:53.243748 29408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:53.245000 29408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:53.254426 29414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:53.254581 29408 server_base.cc:1034] running on GCE node
W20260929 03:09:53.254478 29413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:53.254688 29416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:53.255344 29408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:53.256563 29408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:53.257723 29408 hybrid_clock.cc:648] HybridClock initialized: now 1790651393257700 us; error 44 us; skew 500 ppm
Sep 29 03:09:53 dist-test-slave-bdps krb5kdc[29395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:53.260658 29408 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:09:53.261664 29408 webserver.cc:460] Webserver started at http://127.0.0.1:44741/ using document root <none> and password file <none>
I20260929 03:09:53.262290 29408 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:53.262362 29408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:53.262584 29408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:53.264920 29408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "153460c683b44cd7805c8bc8284af9c3"
format_stamp: "Formatted at 2026-09-29 03:09:53 on dist-test-slave-bdps"
server_key: "5996ed91a13cd56f3e8703816d5edc15"
server_key_iv: "a196d8c23f5436902eee052398be0615"
server_key_version: "encryptionkey@0"
I20260929 03:09:53.265522 29408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "153460c683b44cd7805c8bc8284af9c3"
format_stamp: "Formatted at 2026-09-29 03:09:53 on dist-test-slave-bdps"
server_key: "5996ed91a13cd56f3e8703816d5edc15"
server_key_iv: "a196d8c23f5436902eee052398be0615"
server_key_version: "encryptionkey@0"
I20260929 03:09:53.269598 29408 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 03:09:53.272234 29423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:53.273650 29408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 03:09:53.273772 29408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "153460c683b44cd7805c8bc8284af9c3"
format_stamp: "Formatted at 2026-09-29 03:09:53 on dist-test-slave-bdps"
server_key: "5996ed91a13cd56f3e8703816d5edc15"
server_key_iv: "a196d8c23f5436902eee052398be0615"
server_key_version: "encryptionkey@0"
I20260929 03:09:53.273866 29408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:53.286116 29408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:53.289460 29408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:53.289695 29408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:53.297567 29408 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45307
I20260929 03:09:53.297600 29475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45307 every 8 connection(s)
I20260929 03:09:53.298724 29408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:53.302004 29476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:53.302850 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 29408
I20260929 03:09:53.302994 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:53.303390 8005 external_mini_cluster.cc:1428] Setting key 73bcc7bb8b16ff4514ad29ab4774f63f
I20260929 03:09:53.309070 29476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3: Bootstrap starting.
I20260929 03:09:53.311642 29476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:53.312425 29476 log.cc:824] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:53.313493 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:53.305031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.117:45837 (local address 127.0.0.1:45307)
0929 03:09:53.305735 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:53.305760 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:53.305813 (+ 53us) server_negotiation.cc:407] Connection header received
0929 03:09:53.306652 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:53.306673 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:53.307074 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:53.307421 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:53.307866 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:53.308901 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:53.309945 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:53.310440 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:53.310589 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:53.311554 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:53.311559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:53.311572 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:53.312013 (+ 441us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:53.312071 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:53.312095 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:53.312210 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:53.312423 (+ 213us) server_negotiation.cc:299] Negotiation successful
0929 03:09:53.312572 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":139,"threads_started":1}
I20260929 03:09:53.314625 29476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3: No bootstrap required, opened a new log
I20260929 03:09:53.316565 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:53.318850 29476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153460c683b44cd7805c8bc8284af9c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45307 } }
I20260929 03:09:53.319124 29476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:53.319216 29476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153460c683b44cd7805c8bc8284af9c3, State: Initialized, Role: FOLLOWER
I20260929 03:09:53.319763 29476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153460c683b44cd7805c8bc8284af9c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45307 } }
I20260929 03:09:53.319921 29476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:53.319988 29476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:53.320091 29476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:53.321256 29476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153460c683b44cd7805c8bc8284af9c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45307 } }
I20260929 03:09:53.321691 29476 leader_election.cc:302] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 153460c683b44cd7805c8bc8284af9c3; no voters:
I20260929 03:09:53.322057 29476 leader_election.cc:288] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:53.322113 29486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:53.322329 29486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [term 1 LEADER]: Becoming Leader. State: Replica: 153460c683b44cd7805c8bc8284af9c3, State: Running, Role: LEADER
I20260929 03:09:53.322708 29486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153460c683b44cd7805c8bc8284af9c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45307 } }
I20260929 03:09:53.323370 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:53.318699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.117:48839 (local address 127.0.0.1:45307)
0929 03:09:53.318803 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:53.318807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:53.319147 (+ 340us) server_negotiation.cc:407] Connection header received
0929 03:09:53.319257 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:53.319260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:53.319304 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:53.319375 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:53.319665 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:53.320323 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:53.321420 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:53.321762 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:53.321865 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:53.322846 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:53.322850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:53.322853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:53.322867 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:53.322895 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:53.322907 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:53.322962 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:53.323190 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 03:09:53.323229 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 03:09:53.323660 29476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:53.323966 29488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "153460c683b44cd7805c8bc8284af9c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153460c683b44cd7805c8bc8284af9c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45307 } } }
I20260929 03:09:53.324119 29488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:53.325448 29487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 153460c683b44cd7805c8bc8284af9c3. Latest consensus state: current_term: 1 leader_uuid: "153460c683b44cd7805c8bc8284af9c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153460c683b44cd7805c8bc8284af9c3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45307 } } }
I20260929 03:09:53.325641 29487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:53.326073 29490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:53.329125 29490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:53.335542 29490 catalog_manager.cc:1347] Generated new cluster ID: f87faae181b94efab257c024470da705
I20260929 03:09:53.335605 29490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:53.345319 29490 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:53.346567 29490 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:53.354801 29490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 153460c683b44cd7805c8bc8284af9c3: Generated new TSK 0
I20260929 03:09:53.355643 29490 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:53.359407 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 29408
W20260929 03:09:53.365381 29393 connection.cc:535] client connection to 127.0.0.1:45307 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45307): Connection reset by peer (error 104)
2026-09-29T03:09:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:53 dist-test-slave-bdps krb5kdc[29507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:53 dist-test-slave-bdps krb5kdc[29507](info): set up 2 sockets
Sep 29 03:09:53 dist-test-slave-bdps krb5kdc[29507](info): commencing operation
krb5kdc: starting...
W20260929 03:09:54.322099 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:54 dist-test-slave-bdps krb5kdc[29507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 03:09:54.446772 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43139
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:54.568768 29520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:54.569132 29520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:54.569200 29520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:54.573355 29520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:54.573428 29520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:54.573455 29520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:54.573477 29520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:54.573498 29520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:54.579205 29520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43139
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:54.580686 29520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:54.582086 29520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:54.592339 29525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:54.592366 29528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:54.592571 29520 server_base.cc:1034] running on GCE node
W20260929 03:09:54.592357 29526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:54.593502 29520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:54.594931 29520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:54.596093 29520 hybrid_clock.cc:648] HybridClock initialized: now 1790651394596071 us; error 36 us; skew 500 ppm
Sep 29 03:09:54 dist-test-slave-bdps krb5kdc[29507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651394, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:54.603042 29520 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 03:09:54.604197 29520 webserver.cc:460] Webserver started at http://127.0.0.1:40571/ using document root <none> and password file <none>
I20260929 03:09:54.604858 29520 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:54.604929 29520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:54.605140 29520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:54.607640 29520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "5dff5c8cff2643adb9214a75a4ec2b31"
format_stamp: "Formatted at 2026-09-29 03:09:54 on dist-test-slave-bdps"
server_key: "52510162980e47228d156299d09fcd99"
server_key_iv: "61dc95015d94e1e6ba73fdd74f9df13d"
server_key_version: "encryptionkey@0"
I20260929 03:09:54.608265 29520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "5dff5c8cff2643adb9214a75a4ec2b31"
format_stamp: "Formatted at 2026-09-29 03:09:54 on dist-test-slave-bdps"
server_key: "52510162980e47228d156299d09fcd99"
server_key_iv: "61dc95015d94e1e6ba73fdd74f9df13d"
server_key_version: "encryptionkey@0"
I20260929 03:09:54.612582 29520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260929 03:09:54.615319 29535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:54.616704 29520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 03:09:54.616822 29520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "5dff5c8cff2643adb9214a75a4ec2b31"
format_stamp: "Formatted at 2026-09-29 03:09:54 on dist-test-slave-bdps"
server_key: "52510162980e47228d156299d09fcd99"
server_key_iv: "61dc95015d94e1e6ba73fdd74f9df13d"
server_key_version: "encryptionkey@0"
I20260929 03:09:54.616932 29520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:54.622014 29520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:54.625133 29520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:54.625303 29520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:54.633024 29520 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43139
I20260929 03:09:54.633034 29587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43139 every 8 connection(s)
I20260929 03:09:54.634111 29520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:54.637295 29588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:54.643955 29588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31: Bootstrap starting.
I20260929 03:09:54.644201 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 29520
I20260929 03:09:54.644356 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:54.644675 8005 external_mini_cluster.cc:1428] Setting key 787b2b48b2246d08a73f48b3fab5e7b3
I20260929 03:09:54.646944 29588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:54.647783 29588 log.cc:824] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:54.650018 29588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31: No bootstrap required, opened a new log
I20260929 03:09:54.650342 29591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:54.646155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.117:33655 (local address 127.0.0.1:43139)
0929 03:09:54.646622 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:54.646639 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:54.646679 (+ 40us) server_negotiation.cc:407] Connection header received
0929 03:09:54.647418 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:54.647439 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:54.647797 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:54.648150 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:54.648958 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:54.648963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:54.648976 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:54.648986 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:54.649040 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:54.649060 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:54.649153 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:54.649274 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 03:09:54.649453 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":129,"threads_started":1}
I20260929 03:09:54.653278 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:54.654199 29588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dff5c8cff2643adb9214a75a4ec2b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43139 } }
I20260929 03:09:54.654448 29588 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:54.654505 29588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dff5c8cff2643adb9214a75a4ec2b31, State: Initialized, Role: FOLLOWER
I20260929 03:09:54.655051 29588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dff5c8cff2643adb9214a75a4ec2b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43139 } }
I20260929 03:09:54.655194 29588 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:54.655257 29588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:54.655360 29588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:54.656538 29588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dff5c8cff2643adb9214a75a4ec2b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43139 } }
I20260929 03:09:54.656975 29588 leader_election.cc:302] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dff5c8cff2643adb9214a75a4ec2b31; no voters:
I20260929 03:09:54.657110 29591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:54.655521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.117:38805 (local address 127.0.0.1:43139)
0929 03:09:54.655637 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:54.655641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:54.655859 (+ 218us) server_negotiation.cc:407] Connection header received
0929 03:09:54.655959 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:54.655961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:54.655995 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:54.656058 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:54.656739 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:54.656743 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:54.656746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:54.656758 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:54.656787 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:54.656800 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:54.656884 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:54.656936 (+ 52us) server_negotiation.cc:299] Negotiation successful
0929 03:09:54.656976 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 03:09:54.657377 29588 leader_election.cc:288] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:54.657414 29598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:54.657676 29598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [term 1 LEADER]: Becoming Leader. State: Replica: 5dff5c8cff2643adb9214a75a4ec2b31, State: Running, Role: LEADER
I20260929 03:09:54.658056 29598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dff5c8cff2643adb9214a75a4ec2b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43139 } }
I20260929 03:09:54.658852 29588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:54.660034 29600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dff5c8cff2643adb9214a75a4ec2b31. Latest consensus state: current_term: 1 leader_uuid: "5dff5c8cff2643adb9214a75a4ec2b31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dff5c8cff2643adb9214a75a4ec2b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43139 } } }
I20260929 03:09:54.660069 29599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dff5c8cff2643adb9214a75a4ec2b31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dff5c8cff2643adb9214a75a4ec2b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43139 } } }
I20260929 03:09:54.660157 29600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:54.660164 29599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:54.660591 29607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:54.663894 29607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:54.670053 29607 catalog_manager.cc:1347] Generated new cluster ID: 6da65d226c4a488f8c758102171e3f3a
I20260929 03:09:54.670120 29607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:54.702131 29607 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:54.702551 29607 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:54.709287 29607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dff5c8cff2643adb9214a75a4ec2b31: Generated new TSK 0
I20260929 03:09:54.710163 29607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:54.721071 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 29520
2026-09-29T03:09:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1361 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:54 dist-test-slave-bdps krb5kdc[29619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:54 dist-test-slave-bdps krb5kdc[29619](info): set up 2 sockets
Sep 29 03:09:54 dist-test-slave-bdps krb5kdc[29619](info): commencing operation
krb5kdc: starting...
W20260929 03:09:55.670197 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.929s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:55 dist-test-slave-bdps krb5kdc[29619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:09:55.796991 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:39275
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:42485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:39275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:55.921310 29632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:55.921778 29632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:55.921854 29632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:55.926139 29632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:55.926219 29632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:55.926254 29632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:55.926275 29632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:55.926309 29632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:55.932178 29632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:42485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:39275
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:39275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:55.933516 29632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:55.934799 29632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:55.944257 29637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:55.944376 29638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:55.944496 29632 server_base.cc:1034] running on GCE node
W20260929 03:09:55.944555 29640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:55.945185 29632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:55.946472 29632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:55.947626 29632 hybrid_clock.cc:648] HybridClock initialized: now 1790651395947604 us; error 35 us; skew 500 ppm
Sep 29 03:09:55 dist-test-slave-bdps krb5kdc[29619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651395, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:55.950506 29632 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:09:55.951449 29632 webserver.cc:460] Webserver started at http://127.7.209.126:43007/ using document root <none> and password file <none>
I20260929 03:09:55.952044 29632 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:55.952111 29632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:55.952342 29632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:55.954627 29632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "e874eedcf0074bb6adda5acf1627fd6c"
format_stamp: "Formatted at 2026-09-29 03:09:55 on dist-test-slave-bdps"
server_key: "72addfefdd1b81d74b088cded92bdcf9"
server_key_iv: "50f24cb71a5f58a12c589ceda948ed29"
server_key_version: "encryptionkey@0"
I20260929 03:09:55.955217 29632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "e874eedcf0074bb6adda5acf1627fd6c"
format_stamp: "Formatted at 2026-09-29 03:09:55 on dist-test-slave-bdps"
server_key: "72addfefdd1b81d74b088cded92bdcf9"
server_key_iv: "50f24cb71a5f58a12c589ceda948ed29"
server_key_version: "encryptionkey@0"
I20260929 03:09:55.959219 29632 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:09:55.961949 29647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:55.963410 29632 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:09:55.963538 29632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "e874eedcf0074bb6adda5acf1627fd6c"
format_stamp: "Formatted at 2026-09-29 03:09:55 on dist-test-slave-bdps"
server_key: "72addfefdd1b81d74b088cded92bdcf9"
server_key_iv: "50f24cb71a5f58a12c589ceda948ed29"
server_key_version: "encryptionkey@0"
I20260929 03:09:55.963649 29632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:56.014926 29632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:56.018640 29632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:56.018873 29632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:56.026856 29632 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:39275
I20260929 03:09:56.026889 29699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:39275 every 8 connection(s)
I20260929 03:09:56.027992 29632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:56.031294 29700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:56.034076 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 29632
I20260929 03:09:56.034229 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:56.034603 8005 external_mini_cluster.cc:1428] Setting key 5887f5c5f731abfd6122a6f4f301f6d3
I20260929 03:09:56.038098 29700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c: Bootstrap starting.
I20260929 03:09:56.040652 29700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:56.041491 29700 log.cc:824] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:56.044049 29700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c: No bootstrap required, opened a new log
Sep 29 03:09:56 dist-test-slave-bdps krb5kdc[29619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:09:56.047844 29700 raft_consensus.cc:348] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e874eedcf0074bb6adda5acf1627fd6c" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39275 } }
I20260929 03:09:56.048169 29700 raft_consensus.cc:374] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:56.048236 29700 raft_consensus.cc:729] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e874eedcf0074bb6adda5acf1627fd6c, State: Initialized, Role: FOLLOWER
I20260929 03:09:56.048795 29700 consensus_queue.cc:260] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e874eedcf0074bb6adda5acf1627fd6c" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39275 } }
I20260929 03:09:56.048933 29700 raft_consensus.cc:388] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:56.049005 29700 raft_consensus.cc:482] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:56.049113 29700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:56.050241 29700 raft_consensus.cc:504] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e874eedcf0074bb6adda5acf1627fd6c" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39275 } }
I20260929 03:09:56.050614 29700 leader_election.cc:302] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e874eedcf0074bb6adda5acf1627fd6c; no voters:
I20260929 03:09:56.050743 29703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:56.036175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59897 (local address 127.7.209.126:39275)
0929 03:09:56.036635 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:56.036651 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:56.036690 (+ 39us) server_negotiation.cc:407] Connection header received
0929 03:09:56.037379 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:56.037399 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:56.037821 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:56.038167 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:56.039004 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:56.040076 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:56.041040 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:56.041571 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:56.041757 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:56.044979 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:56.044985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:56.045000 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:56.045028 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:56.047307 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:56.047923 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:56.047929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:56.047934 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:56.048067 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:56.048441 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:56.048444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:56.048445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:56.048891 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:56.049123 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:56.049517 (+ 394us) server_negotiation.cc:299] Negotiation successful
0929 03:09:56.049803 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":140,"threads_started":1}
I20260929 03:09:56.050984 29700 leader_election.cc:288] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:56.051380 29706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:56.051580 29706 raft_consensus.cc:686] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [term 1 LEADER]: Becoming Leader. State: Replica: e874eedcf0074bb6adda5acf1627fd6c, State: Running, Role: LEADER
I20260929 03:09:56.052043 29706 consensus_queue.cc:237] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e874eedcf0074bb6adda5acf1627fd6c" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39275 } }
I20260929 03:09:56.052421 29700 sys_catalog.cc:564] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:56.054675 29705 sys_catalog.cc:455] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e874eedcf0074bb6adda5acf1627fd6c. Latest consensus state: current_term: 1 leader_uuid: "e874eedcf0074bb6adda5acf1627fd6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e874eedcf0074bb6adda5acf1627fd6c" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39275 } } }
I20260929 03:09:56.054688 29707 sys_catalog.cc:455] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e874eedcf0074bb6adda5acf1627fd6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e874eedcf0074bb6adda5acf1627fd6c" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 39275 } } }
I20260929 03:09:56.054827 29707 sys_catalog.cc:458] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:56.055073 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:56.055238 29714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:56.054826 29705 sys_catalog.cc:458] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:56.059051 29714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:56.065827 29714 catalog_manager.cc:1347] Generated new cluster ID: 3946bc59700849339c5732922e6577ea
I20260929 03:09:56.065899 29714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:56.070040 29703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:56.057680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59167 (local address 127.7.209.126:39275)
0929 03:09:56.057811 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:56.057815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:56.061221 (+ 3406us) server_negotiation.cc:407] Connection header received
0929 03:09:56.061362 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:56.061365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:56.061423 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:56.061497 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:56.062350 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:56.062962 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:56.063967 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:56.064280 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:56.064387 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:56.066317 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:56.066321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:56.066324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:09:56.066351 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:56.068144 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:56.068668 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:56.068672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:56.068674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:56.068755 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:09:56.069089 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:09:56.069093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:09:56.069094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:09:56.069308 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:56.069466 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:56.069753 (+ 287us) server_negotiation.cc:299] Negotiation successful
0929 03:09:56.069866 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 03:09:56.074710 29714 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:56.076076 29714 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:56.094319 29714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e874eedcf0074bb6adda5acf1627fd6c: Generated new TSK 0
I20260929 03:09:56.095342 29714 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:56.106191 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 29632
W20260929 03:09:56.112264 29617 connection.cc:535] client connection to 127.7.209.126:39275 recv error: Network error: failed to read from TLS socket (remote: 127.7.209.126:39275): Connection reset by peer (error 104)
2026-09-29T03:09:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:56 dist-test-slave-bdps krb5kdc[29731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:56 dist-test-slave-bdps krb5kdc[29731](info): set up 2 sockets
Sep 29 03:09:56 dist-test-slave-bdps krb5kdc[29731](info): commencing operation
krb5kdc: starting...
W20260929 03:09:57.066944 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:57 dist-test-slave-bdps krb5kdc[29731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:09:57.195595 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:41359
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:41359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:57.318315 29744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:57.318673 29744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:57.318718 29744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:57.322862 29744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:57.322942 29744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:57.322959 29744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:57.322976 29744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:57.322995 29744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:57.328685 29744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:41359
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:41359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:57.330019 29744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:57.331234 29744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:57.340619 29750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:57.340687 29744 server_base.cc:1034] running on GCE node
W20260929 03:09:57.340723 29749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:57.340783 29752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:57.341485 29744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:57.342788 29744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:57.343961 29744 hybrid_clock.cc:648] HybridClock initialized: now 1790651397343943 us; error 29 us; skew 500 ppm
Sep 29 03:09:57 dist-test-slave-bdps krb5kdc[29731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651397, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:57.347036 29744 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:09:57.347998 29744 webserver.cc:460] Webserver started at http://127.7.209.126:36815/ using document root <none> and password file <none>
I20260929 03:09:57.348603 29744 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:57.348682 29744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:57.348906 29744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:57.351292 29744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "dfb128baad554a2ca1522727fa87cd46"
format_stamp: "Formatted at 2026-09-29 03:09:57 on dist-test-slave-bdps"
server_key: "2b6e2d0daa4b489e48f886914db7d6b1"
server_key_iv: "fe1922aff8f6a2bd1ff13b9b27e3bd75"
server_key_version: "encryptionkey@0"
I20260929 03:09:57.351905 29744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "dfb128baad554a2ca1522727fa87cd46"
format_stamp: "Formatted at 2026-09-29 03:09:57 on dist-test-slave-bdps"
server_key: "2b6e2d0daa4b489e48f886914db7d6b1"
server_key_iv: "fe1922aff8f6a2bd1ff13b9b27e3bd75"
server_key_version: "encryptionkey@0"
I20260929 03:09:57.355957 29744 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:09:57.358543 29759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:57.359894 29744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 03:09:57.360018 29744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "dfb128baad554a2ca1522727fa87cd46"
format_stamp: "Formatted at 2026-09-29 03:09:57 on dist-test-slave-bdps"
server_key: "2b6e2d0daa4b489e48f886914db7d6b1"
server_key_iv: "fe1922aff8f6a2bd1ff13b9b27e3bd75"
server_key_version: "encryptionkey@0"
I20260929 03:09:57.360119 29744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:57.384172 29744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:57.387604 29744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:57.387840 29744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:57.395694 29744 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:41359
I20260929 03:09:57.395704 29811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:41359 every 8 connection(s)
I20260929 03:09:57.396811 29744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:57.400175 29812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:57.401779 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 29744
I20260929 03:09:57.401948 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:57.402329 8005 external_mini_cluster.cc:1428] Setting key 01440727806162b462d2acbb679dfc9b
I20260929 03:09:57.407711 29812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46: Bootstrap starting.
I20260929 03:09:57.410485 29812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:57.411314 29812 log.cc:824] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:57.412742 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:57.404121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51435 (local address 127.7.209.126:41359)
0929 03:09:57.404681 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:57.404700 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:57.404745 (+ 45us) server_negotiation.cc:407] Connection header received
0929 03:09:57.405486 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:57.405506 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:57.405981 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:57.406375 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:57.406846 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:57.407987 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:57.408939 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:57.409460 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:57.409659 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:57.410649 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:57.410656 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:57.410677 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:57.411125 (+ 448us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:57.411191 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:57.411216 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:57.411348 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:57.411587 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 03:09:57.411747 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":179,"threads_started":1}
I20260929 03:09:57.413698 29812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46: No bootstrap required, opened a new log
I20260929 03:09:57.415940 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:57.417770 29812 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfb128baad554a2ca1522727fa87cd46" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41359 } }
I20260929 03:09:57.418012 29812 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:57.418291 29812 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfb128baad554a2ca1522727fa87cd46, State: Initialized, Role: FOLLOWER
I20260929 03:09:57.418830 29812 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfb128baad554a2ca1522727fa87cd46" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41359 } }
I20260929 03:09:57.419005 29812 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:57.419077 29812 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:57.419179 29812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:57.420269 29812 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfb128baad554a2ca1522727fa87cd46" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41359 } }
I20260929 03:09:57.420663 29812 leader_election.cc:302] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfb128baad554a2ca1522727fa87cd46; no voters:
I20260929 03:09:57.421007 29812 leader_election.cc:288] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:57.421095 29822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:57.421272 29822 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [term 1 LEADER]: Becoming Leader. State: Replica: dfb128baad554a2ca1522727fa87cd46, State: Running, Role: LEADER
I20260929 03:09:57.422019 29822 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfb128baad554a2ca1522727fa87cd46" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41359 } }
I20260929 03:09:57.422531 29812 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:57.422704 29815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:57.418113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36231 (local address 127.7.209.126:41359)
0929 03:09:57.418226 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:57.418230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:57.418452 (+ 222us) server_negotiation.cc:407] Connection header received
0929 03:09:57.418527 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:57.418529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:57.418578 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:57.418656 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:57.418987 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:57.419633 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:57.420707 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:09:57.421021 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:09:57.421115 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:09:57.422168 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:57.422172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:57.422174 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:57.422187 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:57.422213 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:57.422223 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:57.422420 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:57.422528 (+ 108us) server_negotiation.cc:299] Negotiation successful
0929 03:09:57.422566 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 03:09:57.424070 29823 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfb128baad554a2ca1522727fa87cd46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfb128baad554a2ca1522727fa87cd46" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41359 } } }
I20260929 03:09:57.424221 29823 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:57.424402 29824 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfb128baad554a2ca1522727fa87cd46. Latest consensus state: current_term: 1 leader_uuid: "dfb128baad554a2ca1522727fa87cd46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfb128baad554a2ca1522727fa87cd46" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 41359 } } }
I20260929 03:09:57.424592 29824 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46 [sys.catalog]: This master's current role is: LEADER
W20260929 03:09:57.427706 29836 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:09:57.427773 29836 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:09:57.427866 29837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:57.428565 29837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:57.434818 29837 catalog_manager.cc:1347] Generated new cluster ID: 3c49ca4a45154b4e97ec31beb119704a
I20260929 03:09:57.434892 29837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:57.462926 29837 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:57.464165 29837 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:57.470355 29837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfb128baad554a2ca1522727fa87cd46: Generated new TSK 0
I20260929 03:09:57.471191 29837 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:57.474959 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 29744
W20260929 03:09:57.481076 29729 connection.cc:535] client connection to 127.7.209.126:41359 recv error: Network error: failed to read from TLS socket (remote: 127.7.209.126:41359): Connection reset by peer (error 104)
2026-09-29T03:09:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1368 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:57 dist-test-slave-bdps krb5kdc[29843](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:57 dist-test-slave-bdps krb5kdc[29843](info): set up 2 sockets
Sep 29 03:09:57 dist-test-slave-bdps krb5kdc[29843](info): commencing operation
krb5kdc: starting...
W20260929 03:09:58.426079 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:58 dist-test-slave-bdps krb5kdc[29843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:09:58.552413 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:46621
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:46621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:09:58.677892 29856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:09:58.678239 29856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:09:58.678284 29856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:09:58.682497 29856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:09:58.682590 29856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:09:58.682610 29856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:09:58.682627 29856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:09:58.682658 29856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:09:58.688387 29856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:46621
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:46621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:09:58.689719 29856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:09:58.691002 29856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:09:58.700359 29861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:58.700348 29862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:09:58.700348 29864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:09:58.700584 29856 server_base.cc:1034] running on GCE node
I20260929 03:09:58.701179 29856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:09:58.702416 29856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:09:58.703573 29856 hybrid_clock.cc:648] HybridClock initialized: now 1790651398703558 us; error 26 us; skew 500 ppm
Sep 29 03:09:58 dist-test-slave-bdps krb5kdc[29843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651398, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:09:58.706393 29856 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:09:58.707280 29856 webserver.cc:460] Webserver started at http://127.7.209.126:45521/ using document root <none> and password file <none>
I20260929 03:09:58.707901 29856 fs_manager.cc:359] Metadata directory not provided
I20260929 03:09:58.707981 29856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:09:58.708190 29856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:09:58.710501 29856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "61ade2914ef744feb008ece334baece5"
format_stamp: "Formatted at 2026-09-29 03:09:58 on dist-test-slave-bdps"
server_key: "21878ba1c34b23ff9d4ca74e8c4eca62"
server_key_iv: "1350790dbd2ad6b3d32aa4a7754e464f"
server_key_version: "encryptionkey@0"
I20260929 03:09:58.711046 29856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "61ade2914ef744feb008ece334baece5"
format_stamp: "Formatted at 2026-09-29 03:09:58 on dist-test-slave-bdps"
server_key: "21878ba1c34b23ff9d4ca74e8c4eca62"
server_key_iv: "1350790dbd2ad6b3d32aa4a7754e464f"
server_key_version: "encryptionkey@0"
I20260929 03:09:58.714893 29856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:09:58.717305 29871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:09:58.718595 29856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 03:09:58.718713 29856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "61ade2914ef744feb008ece334baece5"
format_stamp: "Formatted at 2026-09-29 03:09:58 on dist-test-slave-bdps"
server_key: "21878ba1c34b23ff9d4ca74e8c4eca62"
server_key_iv: "1350790dbd2ad6b3d32aa4a7754e464f"
server_key_version: "encryptionkey@0"
I20260929 03:09:58.718798 29856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:09:58.723518 29856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:09:58.726526 29856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:09:58.726676 29856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:09:58.734058 29856 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:46621
I20260929 03:09:58.734090 29923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:46621 every 8 connection(s)
I20260929 03:09:58.735051 29856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:09:58.738154 29924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:09:58.738435 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 29856
I20260929 03:09:58.738582 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:09:58.738933 8005 external_mini_cluster.cc:1428] Setting key 0bada18be96109d5b7668d64a664e048
I20260929 03:09:58.744493 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:58.740456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60079 (local address 127.7.209.126:46621)
0929 03:09:58.740914 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:58.740931 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:58.740970 (+ 39us) server_negotiation.cc:407] Connection header received
0929 03:09:58.741749 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:58.741771 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:58.742090 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:58.742437 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:58.743251 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:58.743256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:58.743269 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:58.743281 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:58.743338 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:58.743358 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:58.743495 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:58.743569 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 03:09:58.743738 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":154,"threads_started":1}
I20260929 03:09:58.745337 29924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5: Bootstrap starting.
I20260929 03:09:58.747334 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:09:58.748013 29924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5: Neither blocks nor log segments found. Creating new log.
I20260929 03:09:58.748832 29924 log.cc:824] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5: Log is configured to *not* fsync() on all Append() calls
I20260929 03:09:58.751022 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:09:58.749651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34919 (local address 127.7.209.126:46621)
0929 03:09:58.749763 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 03:09:58.749766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:09:58.749811 (+ 45us) server_negotiation.cc:407] Connection header received
0929 03:09:58.749936 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:09:58.749939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:09:58.749965 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:09:58.750025 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:09:58.750669 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:09:58.750672 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:09:58.750674 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:09:58.750683 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:09:58.750724 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:09:58.750734 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:09:58.750799 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 03:09:58.750891 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 03:09:58.750920 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:09:58.751507 29924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5: No bootstrap required, opened a new log
I20260929 03:09:58.755298 29924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61ade2914ef744feb008ece334baece5" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46621 } }
I20260929 03:09:58.755578 29924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:09:58.755647 29924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61ade2914ef744feb008ece334baece5, State: Initialized, Role: FOLLOWER
I20260929 03:09:58.756145 29924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61ade2914ef744feb008ece334baece5" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46621 } }
I20260929 03:09:58.756292 29924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:09:58.756359 29924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:09:58.756461 29924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:09:58.757400 29924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61ade2914ef744feb008ece334baece5" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46621 } }
I20260929 03:09:58.757803 29924 leader_election.cc:302] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61ade2914ef744feb008ece334baece5; no voters:
I20260929 03:09:58.758112 29924 leader_election.cc:288] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:09:58.758232 29934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:09:58.758457 29934 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [term 1 LEADER]: Becoming Leader. State: Replica: 61ade2914ef744feb008ece334baece5, State: Running, Role: LEADER
I20260929 03:09:58.758839 29934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61ade2914ef744feb008ece334baece5" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46621 } }
I20260929 03:09:58.759509 29924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:09:58.760790 29936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61ade2914ef744feb008ece334baece5. Latest consensus state: current_term: 1 leader_uuid: "61ade2914ef744feb008ece334baece5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61ade2914ef744feb008ece334baece5" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46621 } } }
I20260929 03:09:58.760802 29935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61ade2914ef744feb008ece334baece5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61ade2914ef744feb008ece334baece5" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 46621 } } }
I20260929 03:09:58.760928 29936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:58.760928 29935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5 [sys.catalog]: This master's current role is: LEADER
I20260929 03:09:58.761343 29943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:09:58.765089 29943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:09:58.771879 29943 catalog_manager.cc:1347] Generated new cluster ID: 4ff16b88972a4a14bf1aecacb9445a5d
I20260929 03:09:58.772015 29943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:09:58.790602 29943 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:09:58.791114 29943 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:09:58.802515 29943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61ade2914ef744feb008ece334baece5: Generated new TSK 0
I20260929 03:09:58.803500 29943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:09:58.818323 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 29856
2026-09-29T03:09:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:09:58 dist-test-slave-bdps krb5kdc[29955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:09:58 dist-test-slave-bdps krb5kdc[29955](info): set up 2 sockets
Sep 29 03:09:58 dist-test-slave-bdps krb5kdc[29955](info): commencing operation
krb5kdc: starting...
W20260929 03:09:59.766289 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:09:59 dist-test-slave-bdps krb5kdc[29955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:09:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:09:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:09:59.890192 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:43399
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:43399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:10:00.017817 29968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:10:00.018169 29968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:10:00.018244 29968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:10:00.022392 29968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:10:00.022469 29968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:10:00.022496 29968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:10:00.022518 29968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:10:00.022545 29968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:10:00.028182 29968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:43399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:43399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:10:00.029533 29968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:10:00.030907 29968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:10:00.040546 29973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:00.040527 29974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:00.040521 29976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:10:00.040987 29968 server_base.cc:1034] running on GCE node
I20260929 03:10:00.041668 29968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:10:00.043035 29968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:10:00.044214 29968 hybrid_clock.cc:648] HybridClock initialized: now 1790651400044195 us; error 39 us; skew 500 ppm
Sep 29 03:10:00 dist-test-slave-bdps krb5kdc[29955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651400, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:10:00.053944 29968 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:10:00.055209 29968 webserver.cc:460] Webserver started at http://127.7.209.126:36483/ using document root <none> and password file <none>
I20260929 03:10:00.055909 29968 fs_manager.cc:359] Metadata directory not provided
I20260929 03:10:00.055985 29968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:10:00.056236 29968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:10:00.058818 29968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "542d23fe168e419d9b3771a60336b3d6"
format_stamp: "Formatted at 2026-09-29 03:10:00 on dist-test-slave-bdps"
server_key: "7e50dca65b955bc8ba43c49ef6e2fb68"
server_key_iv: "1bc1f181345eb8a40b3a3100b915ee43"
server_key_version: "encryptionkey@0"
I20260929 03:10:00.059434 29968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "542d23fe168e419d9b3771a60336b3d6"
format_stamp: "Formatted at 2026-09-29 03:10:00 on dist-test-slave-bdps"
server_key: "7e50dca65b955bc8ba43c49ef6e2fb68"
server_key_iv: "1bc1f181345eb8a40b3a3100b915ee43"
server_key_version: "encryptionkey@0"
I20260929 03:10:00.063730 29968 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:10:00.066377 29983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:10:00.067835 29968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:10:00.067955 29968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "542d23fe168e419d9b3771a60336b3d6"
format_stamp: "Formatted at 2026-09-29 03:10:00 on dist-test-slave-bdps"
server_key: "7e50dca65b955bc8ba43c49ef6e2fb68"
server_key_iv: "1bc1f181345eb8a40b3a3100b915ee43"
server_key_version: "encryptionkey@0"
I20260929 03:10:00.068053 29968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:10:00.100572 29968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:10:00.104403 29968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:10:00.104612 29968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:10:00.113029 29968 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:43399
I20260929 03:10:00.113056 30035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:43399 every 8 connection(s)
I20260929 03:10:00.114382 29968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:10:00.117465 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 29968
I20260929 03:10:00.117653 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:10:00.118048 8005 external_mini_cluster.cc:1428] Setting key 547af68c71bf71e29069eeb4dcc8d142
I20260929 03:10:00.118079 30036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:10:00.125718 30036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6: Bootstrap starting.
Sep 29 03:10:00 dist-test-slave-bdps krb5kdc[29955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:10:00.128866 30036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6: Neither blocks nor log segments found. Creating new log.
I20260929 03:10:00.129737 30036 log.cc:824] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6: Log is configured to *not* fsync() on all Append() calls
I20260929 03:10:00.132059 30036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6: No bootstrap required, opened a new log
I20260929 03:10:00.133904 30039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:00.119828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58841 (local address 127.7.209.126:43399)
0929 03:10:00.120358 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:00.120381 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:00.120425 (+ 44us) server_negotiation.cc:407] Connection header received
0929 03:10:00.121325 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:00.121348 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:00.121824 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:00.122211 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:00.123154 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:00.124265 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:00.125207 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:00.125776 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:00.125947 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:00.129017 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:00.129022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:00.129036 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:00.129065 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:00.130935 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:00.131402 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:00.131406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:00.131411 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:00.131520 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:00.131810 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:00.131812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:00.131813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:00.132255 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:00.132413 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:00.132715 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 03:10:00.133015 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":169,"threads_started":1}
I20260929 03:10:00.136301 30036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542d23fe168e419d9b3771a60336b3d6" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43399 } }
I20260929 03:10:00.136544 30036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:10:00.136605 30036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 542d23fe168e419d9b3771a60336b3d6, State: Initialized, Role: FOLLOWER
I20260929 03:10:00.137032 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:10:00.137094 30036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542d23fe168e419d9b3771a60336b3d6" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43399 } }
I20260929 03:10:00.137257 30036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:10:00.137315 30036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:10:00.137408 30036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:10:00.138594 30036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542d23fe168e419d9b3771a60336b3d6" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43399 } }
I20260929 03:10:00.139007 30036 leader_election.cc:302] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 542d23fe168e419d9b3771a60336b3d6; no voters:
I20260929 03:10:00.139350 30036 leader_election.cc:288] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:10:00.139420 30045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:10:00.139665 30045 raft_consensus.cc:686] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [term 1 LEADER]: Becoming Leader. State: Replica: 542d23fe168e419d9b3771a60336b3d6, State: Running, Role: LEADER
I20260929 03:10:00.140043 30045 consensus_queue.cc:237] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542d23fe168e419d9b3771a60336b3d6" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43399 } }
I20260929 03:10:00.140754 30036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:10:00.141352 30045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "542d23fe168e419d9b3771a60336b3d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542d23fe168e419d9b3771a60336b3d6" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43399 } } }
I20260929 03:10:00.141538 30045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [sys.catalog]: This master's current role is: LEADER
I20260929 03:10:00.141829 30046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 542d23fe168e419d9b3771a60336b3d6. Latest consensus state: current_term: 1 leader_uuid: "542d23fe168e419d9b3771a60336b3d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "542d23fe168e419d9b3771a60336b3d6" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 43399 } } }
I20260929 03:10:00.141985 30046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6 [sys.catalog]: This master's current role is: LEADER
I20260929 03:10:00.142166 30054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:10:00.146118 30054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:10:00.152448 30039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:00.139885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50837 (local address 127.7.209.126:43399)
0929 03:10:00.139991 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:00.139994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:00.143126 (+ 3132us) server_negotiation.cc:407] Connection header received
0929 03:10:00.143341 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:00.143343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:00.143423 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:00.143521 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:00.144426 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:00.145209 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:00.146435 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:00.146772 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:00.146848 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:00.148745 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:00.148749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:00.148752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:00.148776 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:00.150494 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:00.150986 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:00.150990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:00.150992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:00.151080 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:00.151404 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:00.151408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:00.151410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:00.151631 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:00.151741 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:00.152125 (+ 384us) server_negotiation.cc:299] Negotiation successful
0929 03:10:00.152244 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 03:10:00.153484 30054 catalog_manager.cc:1347] Generated new cluster ID: 72d9ef23ee8f4d8a803d388e3ba637d5
I20260929 03:10:00.153545 30054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:10:00.175576 30054 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:10:00.176640 30054 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:10:00.193060 30054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 542d23fe168e419d9b3771a60336b3d6: Generated new TSK 0
I20260929 03:10:00.194031 30054 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:10:00.198000 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 29968
2026-09-29T03:10:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:10:00 dist-test-slave-bdps krb5kdc[30067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:10:00 dist-test-slave-bdps krb5kdc[30067](info): set up 2 sockets
Sep 29 03:10:00 dist-test-slave-bdps krb5kdc[30067](info): commencing operation
krb5kdc: starting...
W20260929 03:10:01.158151 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.939s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:10:01 dist-test-slave-bdps krb5kdc[30067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:10:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:10:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:10:01.282688 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:38257
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:33605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:38257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:10:01.408640 30080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:10:01.409022 30080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:10:01.409073 30080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:10:01.413318 30080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:10:01.413390 30080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:10:01.413410 30080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:10:01.413426 30080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:10:01.413441 30080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:10:01.419414 30080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:33605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:38257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:38257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:10:01.420759 30080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:10:01.422096 30080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:10:01.431789 30086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:01.431782 30088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:10:01.431960 30080 server_base.cc:1034] running on GCE node
W20260929 03:10:01.431813 30085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:10:01.432680 30080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:10:01.433967 30080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:10:01.435124 30080 hybrid_clock.cc:648] HybridClock initialized: now 1790651401435108 us; error 31 us; skew 500 ppm
Sep 29 03:10:01 dist-test-slave-bdps krb5kdc[30067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651401, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:10:01.438035 30080 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:10:01.438990 30080 webserver.cc:460] Webserver started at http://127.7.209.126:37581/ using document root <none> and password file <none>
I20260929 03:10:01.439597 30080 fs_manager.cc:359] Metadata directory not provided
I20260929 03:10:01.439663 30080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:10:01.439869 30080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:10:01.442204 30080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "612e35b0bbc542c9b60707e7c23fde62"
format_stamp: "Formatted at 2026-09-29 03:10:01 on dist-test-slave-bdps"
server_key: "a9753da345855683b3cc8067ab78dfd5"
server_key_iv: "0c920a9fa891a5ffcc95a2db9b0b941b"
server_key_version: "encryptionkey@0"
I20260929 03:10:01.442824 30080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "612e35b0bbc542c9b60707e7c23fde62"
format_stamp: "Formatted at 2026-09-29 03:10:01 on dist-test-slave-bdps"
server_key: "a9753da345855683b3cc8067ab78dfd5"
server_key_iv: "0c920a9fa891a5ffcc95a2db9b0b941b"
server_key_version: "encryptionkey@0"
I20260929 03:10:01.447041 30080 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:10:01.449671 30095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:10:01.451025 30080 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 03:10:01.451141 30080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "612e35b0bbc542c9b60707e7c23fde62"
format_stamp: "Formatted at 2026-09-29 03:10:01 on dist-test-slave-bdps"
server_key: "a9753da345855683b3cc8067ab78dfd5"
server_key_iv: "0c920a9fa891a5ffcc95a2db9b0b941b"
server_key_version: "encryptionkey@0"
I20260929 03:10:01.451246 30080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:10:01.474085 30080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:10:01.477419 30080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:10:01.477654 30080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:10:01.485644 30080 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:38257
I20260929 03:10:01.485684 30147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:38257 every 8 connection(s)
I20260929 03:10:01.486819 30080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:10:01.489408 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 30080
I20260929 03:10:01.489549 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:10:01.489903 8005 external_mini_cluster.cc:1428] Setting key 835f17896faf7ca999e6aa4d8152f5ff
I20260929 03:10:01.490360 30148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:10:01.497527 30148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62: Bootstrap starting.
I20260929 03:10:01.499902 30151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:01.491521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45935 (local address 127.7.209.126:38257)
0929 03:10:01.492011 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:01.492028 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:01.492069 (+ 41us) server_negotiation.cc:407] Connection header received
0929 03:10:01.492820 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:01.492840 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:01.493261 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:01.493672 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:01.494165 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:01.495264 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:01.496300 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:01.496817 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:01.496968 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:01.498010 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:01.498015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:01.498031 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:10:01.498483 (+ 452us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:01.498547 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:10:01.498572 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:01.498658 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:01.498813 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 03:10:01.499048 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":174,"threads_started":1}
I20260929 03:10:01.500326 30148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62: Neither blocks nor log segments found. Creating new log.
I20260929 03:10:01.501142 30148 log.cc:824] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62: Log is configured to *not* fsync() on all Append() calls
I20260929 03:10:01.503139 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:10:01.503429 30148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62: No bootstrap required, opened a new log
I20260929 03:10:01.507360 30148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "612e35b0bbc542c9b60707e7c23fde62" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38257 } }
I20260929 03:10:01.507592 30148 raft_consensus.cc:374] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:10:01.507665 30148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 612e35b0bbc542c9b60707e7c23fde62, State: Initialized, Role: FOLLOWER
I20260929 03:10:01.508191 30148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "612e35b0bbc542c9b60707e7c23fde62" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38257 } }
I20260929 03:10:01.508373 30148 raft_consensus.cc:388] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:10:01.508450 30148 raft_consensus.cc:482] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:10:01.508564 30148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:10:01.508939 30151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:01.505267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55963 (local address 127.7.209.126:38257)
0929 03:10:01.505359 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:01.505362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:01.505571 (+ 209us) server_negotiation.cc:407] Connection header received
0929 03:10:01.505705 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:01.505709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:01.505753 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:01.505815 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:01.506069 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:01.506587 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:01.507375 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:01.507603 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:01.507703 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:01.508529 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:01.508533 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:01.508536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:10:01.508548 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:01.508576 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:10:01.508587 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:01.508654 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:01.508771 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 03:10:01.508817 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 03:10:01.509641 30148 raft_consensus.cc:504] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "612e35b0bbc542c9b60707e7c23fde62" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38257 } }
I20260929 03:10:01.510069 30148 leader_election.cc:302] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 612e35b0bbc542c9b60707e7c23fde62; no voters:
I20260929 03:10:01.510448 30148 leader_election.cc:288] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:10:01.510535 30158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:10:01.510722 30158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [term 1 LEADER]: Becoming Leader. State: Replica: 612e35b0bbc542c9b60707e7c23fde62, State: Running, Role: LEADER
I20260929 03:10:01.511087 30158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "612e35b0bbc542c9b60707e7c23fde62" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38257 } }
I20260929 03:10:01.511765 30148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:10:01.512889 30159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "612e35b0bbc542c9b60707e7c23fde62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "612e35b0bbc542c9b60707e7c23fde62" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38257 } } }
I20260929 03:10:01.512882 30160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 612e35b0bbc542c9b60707e7c23fde62. Latest consensus state: current_term: 1 leader_uuid: "612e35b0bbc542c9b60707e7c23fde62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "612e35b0bbc542c9b60707e7c23fde62" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38257 } } }
I20260929 03:10:01.513023 30159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [sys.catalog]: This master's current role is: LEADER
I20260929 03:10:01.513023 30160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62 [sys.catalog]: This master's current role is: LEADER
I20260929 03:10:01.513593 30167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:10:01.516733 30167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:10:01.522434 30167 catalog_manager.cc:1347] Generated new cluster ID: 053d0350eac24dae9a51233ebed880cd
I20260929 03:10:01.522502 30167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:10:01.527961 30167 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:10:01.529275 30167 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:10:01.540066 30167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 612e35b0bbc542c9b60707e7c23fde62: Generated new TSK 0
I20260929 03:10:01.540958 30167 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:10:01.550441 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 30080
2026-09-29T03:10:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1352 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:10:01 dist-test-slave-bdps krb5kdc[30179](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:10:01 dist-test-slave-bdps krb5kdc[30179](info): set up 2 sockets
Sep 29 03:10:01 dist-test-slave-bdps krb5kdc[30179](info): commencing operation
krb5kdc: starting...
W20260929 03:10:02.494117 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:10:02 dist-test-slave-bdps krb5kdc[30179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:10:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:10:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:10:02.624797 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:37585
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:41713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:37585
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:10:02.754056 30192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:10:02.754381 30192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:10:02.754426 30192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:10:02.758496 30192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:10:02.758558 30192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:10:02.758587 30192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:10:02.758616 30192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:10:02.758642 30192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:10:02.764099 30192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:41713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:37585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:37585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:10:02.765483 30192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:10:02.766889 30192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:10:02.777029 30200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:02.777029 30198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:02.777029 30197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:10:02.777338 30192 server_base.cc:1034] running on GCE node
I20260929 03:10:02.778072 30192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:10:02.779371 30192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:10:02.780532 30192 hybrid_clock.cc:648] HybridClock initialized: now 1790651402780516 us; error 36 us; skew 500 ppm
Sep 29 03:10:02 dist-test-slave-bdps krb5kdc[30179](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651402, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:10:02.783510 30192 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:10:02.784441 30192 webserver.cc:460] Webserver started at http://127.7.209.126:34015/ using document root <none> and password file <none>
I20260929 03:10:02.785041 30192 fs_manager.cc:359] Metadata directory not provided
I20260929 03:10:02.785100 30192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:10:02.785296 30192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:10:02.787590 30192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "6d4845818a31463384b370ff18492fd8"
format_stamp: "Formatted at 2026-09-29 03:10:02 on dist-test-slave-bdps"
server_key: "0a0168e017cfaf6eccaed3e335fce848"
server_key_iv: "18d07edd8444f336aaba182fca0f974f"
server_key_version: "encryptionkey@0"
I20260929 03:10:02.788165 30192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "6d4845818a31463384b370ff18492fd8"
format_stamp: "Formatted at 2026-09-29 03:10:02 on dist-test-slave-bdps"
server_key: "0a0168e017cfaf6eccaed3e335fce848"
server_key_iv: "18d07edd8444f336aaba182fca0f974f"
server_key_version: "encryptionkey@0"
I20260929 03:10:02.792182 30192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:10:02.794700 30207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:10:02.796173 30192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:10:02.796288 30192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "6d4845818a31463384b370ff18492fd8"
format_stamp: "Formatted at 2026-09-29 03:10:02 on dist-test-slave-bdps"
server_key: "0a0168e017cfaf6eccaed3e335fce848"
server_key_iv: "18d07edd8444f336aaba182fca0f974f"
server_key_version: "encryptionkey@0"
I20260929 03:10:02.796376 30192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:10:02.801239 30192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:10:02.804283 30192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:10:02.804440 30192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:10:02.811944 30192 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:37585
I20260929 03:10:02.811939 30259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:37585 every 8 connection(s)
I20260929 03:10:02.813005 30192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:10:02.816162 30260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:10:02.822247 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 30192
I20260929 03:10:02.822372 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:10:02.822566 30260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8: Bootstrap starting.
I20260929 03:10:02.822672 8005 external_mini_cluster.cc:1428] Setting key 202b42ca3de58544e684f9c91fd6c262
I20260929 03:10:02.825326 30260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8: Neither blocks nor log segments found. Creating new log.
I20260929 03:10:02.826143 30260 log.cc:824] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8: Log is configured to *not* fsync() on all Append() calls
I20260929 03:10:02.828317 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:02.824100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56915 (local address 127.7.209.126:37585)
0929 03:10:02.824559 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:02.824575 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:02.824614 (+ 39us) server_negotiation.cc:407] Connection header received
0929 03:10:02.825285 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:02.825304 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:02.825709 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:02.826056 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:02.826846 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:02.826851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:02.826864 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:10:02.826874 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:02.826935 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:10:02.826956 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:02.827048 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:02.827209 (+ 161us) server_negotiation.cc:299] Negotiation successful
0929 03:10:02.827376 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":146,"threads_started":1}
I20260929 03:10:02.828591 30260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8: No bootstrap required, opened a new log
I20260929 03:10:02.831135 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:10:02.832558 30260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4845818a31463384b370ff18492fd8" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37585 } }
I20260929 03:10:02.832870 30260 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 03:10:02.832939 30260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d4845818a31463384b370ff18492fd8, State: Initialized, Role: FOLLOWER
I20260929 03:10:02.833422 30260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4845818a31463384b370ff18492fd8" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37585 } }
I20260929 03:10:02.833575 30260 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 03:10:02.833684 30260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 03:10:02.833822 30260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:10:02.834860 30260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4845818a31463384b370ff18492fd8" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37585 } }
I20260929 03:10:02.835216 30260 leader_election.cc:302] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d4845818a31463384b370ff18492fd8; no voters:
I20260929 03:10:02.835337 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:02.833686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53095 (local address 127.7.209.126:37585)
0929 03:10:02.833800 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:02.833804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:02.834055 (+ 251us) server_negotiation.cc:407] Connection header received
0929 03:10:02.834180 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:02.834182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:02.834220 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:02.834280 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:02.834929 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:02.834932 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:02.834934 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 03:10:02.834943 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:02.834969 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 03:10:02.834982 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:02.835061 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:02.835138 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 03:10:02.835174 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 03:10:02.835563 30260 leader_election.cc:288] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 03:10:02.835625 30270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:10:02.835850 30270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [term 1 LEADER]: Becoming Leader. State: Replica: 6d4845818a31463384b370ff18492fd8, State: Running, Role: LEADER
I20260929 03:10:02.836201 30270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4845818a31463384b370ff18492fd8" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37585 } }
I20260929 03:10:02.836894 30260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:10:02.837968 30272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d4845818a31463384b370ff18492fd8. Latest consensus state: current_term: 1 leader_uuid: "6d4845818a31463384b370ff18492fd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4845818a31463384b370ff18492fd8" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37585 } } }
I20260929 03:10:02.838092 30272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [sys.catalog]: This master's current role is: LEADER
I20260929 03:10:02.838434 30271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d4845818a31463384b370ff18492fd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4845818a31463384b370ff18492fd8" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 37585 } } }
I20260929 03:10:02.838482 30279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:10:02.838541 30271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8 [sys.catalog]: This master's current role is: LEADER
I20260929 03:10:02.842077 30279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:10:02.848974 30279 catalog_manager.cc:1347] Generated new cluster ID: e006e47299b0433fb9f1c4a2d6dea8bb
I20260929 03:10:02.849063 30279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:10:02.859468 30279 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:10:02.859848 30279 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:10:02.877661 30279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d4845818a31463384b370ff18492fd8: Generated new TSK 0
I20260929 03:10:02.878984 30279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 03:10:02.884475 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 30192
2026-09-29T03:10:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1333 ms)
[----------] 18 tests from AuthTokenIssuingTest (24378 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:10:02 dist-test-slave-bdps krb5kdc[30291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:10:02 dist-test-slave-bdps krb5kdc[30291](info): set up 2 sockets
Sep 29 03:10:02 dist-test-slave-bdps krb5kdc[30291](info): commencing operation
krb5kdc: starting...
W20260929 03:10:03.838030 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:10:03 dist-test-slave-bdps krb5kdc[30291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:10:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:10:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:10:03.966174 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:40515
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:40515,127.7.209.125:34031,127.7.209.124:45741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:10:04.088518 30304 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 03:10:04.088927 30304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:10:04.088971 30304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:10:04.089041 30304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:10:04.093290 30304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:10:04.093360 30304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:10:04.093387 30304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:10:04.093411 30304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:10:04.093441 30304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:10:04.099050 30304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:40515,127.7.209.125:34031,127.7.209.124:45741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:40515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:10:04.100378 30304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:10:04.101681 30304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:10:04.111184 30310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:04.111145 30312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:04.111156 30309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:10:04.111616 30304 server_base.cc:1034] running on GCE node
I20260929 03:10:04.112288 30304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:10:04.113690 30304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:10:04.114866 30304 hybrid_clock.cc:648] HybridClock initialized: now 1790651404114846 us; error 33 us; skew 500 ppm
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:10:04.117965 30304 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:10:04.118958 30304 webserver.cc:460] Webserver started at http://127.7.209.126:38645/ using document root <none> and password file <none>
I20260929 03:10:04.119582 30304 fs_manager.cc:359] Metadata directory not provided
I20260929 03:10:04.119643 30304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:10:04.119854 30304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:10:04.122267 30304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "492b4abcab744ca199404e9c0c280f18"
format_stamp: "Formatted at 2026-09-29 03:10:04 on dist-test-slave-bdps"
server_key: "d398bc1190fb5b350967d23604dff8d9"
server_key_iv: "31c9705fa42680c13382b51b2c9ef9ba"
server_key_version: "encryptionkey@0"
I20260929 03:10:04.122853 30304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "492b4abcab744ca199404e9c0c280f18"
format_stamp: "Formatted at 2026-09-29 03:10:04 on dist-test-slave-bdps"
server_key: "d398bc1190fb5b350967d23604dff8d9"
server_key_iv: "31c9705fa42680c13382b51b2c9ef9ba"
server_key_version: "encryptionkey@0"
I20260929 03:10:04.127213 30304 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:10:04.129978 30319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:10:04.131366 30304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:10:04.131481 30304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "492b4abcab744ca199404e9c0c280f18"
format_stamp: "Formatted at 2026-09-29 03:10:04 on dist-test-slave-bdps"
server_key: "d398bc1190fb5b350967d23604dff8d9"
server_key_iv: "31c9705fa42680c13382b51b2c9ef9ba"
server_key_version: "encryptionkey@0"
I20260929 03:10:04.131589 30304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:10:04.144285 30304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:10:04.148231 30304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:10:04.148425 30304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:10:04.156504 30304 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:40515
I20260929 03:10:04.156761 30371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:40515 every 8 connection(s)
I20260929 03:10:04.157629 30304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:10:04.160954 30372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:10:04.163561 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 30304
I20260929 03:10:04.163710 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:10:04.164079 8005 external_mini_cluster.cc:1428] Setting key f9b2963bbad1711f234df81c2ef5d2f3
I20260929 03:10:04.166229 30372 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.209.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.125@KRBTEST.COM" created.
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:10:04.183413 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.169207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40604 (local address 127.7.209.126:40515)
0929 03:10:04.169817 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.169837 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.170156 (+ 319us) server_negotiation.cc:407] Connection header received
0929 03:10:04.171349 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.171355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.171643 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.171883 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.172787 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.174251 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.175431 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.175823 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.175946 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.178563 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.178569 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.178580 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.178600 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.180151 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.180600 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.180606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.180607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.180716 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.181006 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.181009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.181011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.181418 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.181539 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.181937 (+ 398us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.182197 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":409,"spinlock_wait_cycles":73472,"thread_start_us":178,"threads_started":1}
I20260929 03:10:04.183413 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.169207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:40515 (local address 127.0.0.1:40604)
0929 03:10:04.169968 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.170022 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:04.170988 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:04.171861 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:04.171866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:04.172225 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:04.172635 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.172650 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.174353 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.174356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.175325 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.175335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.175998 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.176001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.176099 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.176772 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:04.176789 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:04.178421 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:04.180257 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.180266 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.180278 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.180500 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.180805 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.180809 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.180811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.180924 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.181522 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:04.181529 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:04.181701 (+ 172us) client_negotiation.cc:769] Sending connection context
0929 03:10:04.181864 (+ 163us) client_negotiation.cc:240] Negotiation successful
0929 03:10:04.182265 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":134,"thread_start_us":178,"threads_started":1}
I20260929 03:10:04.186058 30372 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:10:04.186802 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.186550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:34031 (local address 127.0.0.1:58744)
0929 03:10:04.186659 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.186730 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 03:10:04.186985 30322 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.209.125:34031: connect: Connection refused (error 111)
I20260929 03:10:04.187803 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.187602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:34031 (local address 127.0.0.1:58756)
0929 03:10:04.187703 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.187749 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 03:10:04.187913 30372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.125:34031: Network error: Client connection negotiation failed: client connection to 127.7.209.125:34031: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.125.keytab.
Entry for principal kudu/127.7.209.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.125.keytab.
I20260929 03:10:04.190673 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.125:34031
--webserver_interface=127.7.209.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.125.keytab
--principal=kudu/127.7.209.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:40515,127.7.209.125:34031,127.7.209.124:45741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:10:04.237135 30372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } attempt: 1
I20260929 03:10:04.238204 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.237839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:34031 (local address 127.0.0.1:58758)
0929 03:10:04.238020 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.238099 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 03:10:04.239012 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.238812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:34031 (local address 127.0.0.1:58764)
0929 03:10:04.238899 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.238946 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 03:10:04.239140 30372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.125:34031: Network error: Client connection negotiation failed: client connection to 127.7.209.125:34031: connect: Connection refused (error 111)
I20260929 03:10:04.307288 30372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } attempt: 2
I20260929 03:10:04.308226 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.307890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:34031 (local address 127.0.0.1:58780)
0929 03:10:04.308058 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.308132 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 03:10:04.308996 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.308808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:34031 (local address 127.0.0.1:58792)
0929 03:10:04.308892 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.308937 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:34031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 03:10:04.309122 30372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.125:34031: Network error: Client connection negotiation failed: client connection to 127.7.209.125:34031: connect: Connection refused (error 111)
W20260929 03:10:04.315807 30379 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 03:10:04.316071 30379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:10:04.316108 30379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:10:04.316164 30379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:10:04.320190 30379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:10:04.320246 30379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:10:04.320276 30379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:10:04.320304 30379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:10:04.320333 30379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:10:04.325543 30379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:40515,127.7.209.125:34031,127.7.209.124:45741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.125.keytab
--principal=kudu/127.7.209.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.125:34031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.209.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:10:04.326825 30379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:10:04.328024 30379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:10:04.337575 30384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:04.337592 30387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:04.337590 30385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:10:04.337848 30379 server_base.cc:1034] running on GCE node
I20260929 03:10:04.338587 30379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:10:04.339862 30379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:10:04.341032 30379 hybrid_clock.cc:648] HybridClock initialized: now 1790651404340999 us; error 40 us; skew 500 ppm
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:10:04.344118 30379 init.cc:377] Logged in from keytab as kudu/127.7.209.125@KRBTEST.COM (short username kudu)
I20260929 03:10:04.345103 30379 webserver.cc:460] Webserver started at http://127.7.209.125:38601/ using document root <none> and password file <none>
I20260929 03:10:04.345777 30379 fs_manager.cc:359] Metadata directory not provided
I20260929 03:10:04.345851 30379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:10:04.346078 30379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:10:04.348481 30379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/data/instance:
uuid: "bfbdcf50917f463f855bf0af04da808a"
format_stamp: "Formatted at 2026-09-29 03:10:04 on dist-test-slave-bdps"
server_key: "24d7dcaf82dbedbddd789fdd60f8527b"
server_key_iv: "879c4f71e14259f2c371bf316bd37e33"
server_key_version: "encryptionkey@0"
I20260929 03:10:04.349090 30379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/wal/instance:
uuid: "bfbdcf50917f463f855bf0af04da808a"
format_stamp: "Formatted at 2026-09-29 03:10:04 on dist-test-slave-bdps"
server_key: "24d7dcaf82dbedbddd789fdd60f8527b"
server_key_iv: "879c4f71e14259f2c371bf316bd37e33"
server_key_version: "encryptionkey@0"
I20260929 03:10:04.353348 30379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260929 03:10:04.356170 30394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:10:04.357738 30379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 03:10:04.357860 30379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/wal
uuid: "bfbdcf50917f463f855bf0af04da808a"
format_stamp: "Formatted at 2026-09-29 03:10:04 on dist-test-slave-bdps"
server_key: "24d7dcaf82dbedbddd789fdd60f8527b"
server_key_iv: "879c4f71e14259f2c371bf316bd37e33"
server_key_version: "encryptionkey@0"
I20260929 03:10:04.357968 30379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:10:04.384027 30379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:10:04.387481 30379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:10:04.387710 30379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:10:04.395886 30379 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.125:34031
I20260929 03:10:04.396167 30446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.125:34031 every 8 connection(s)
I20260929 03:10:04.397054 30379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
I20260929 03:10:04.397560 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 30379
I20260929 03:10:04.397716 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-1/wal/instance
I20260929 03:10:04.398061 8005 external_mini_cluster.cc:1428] Setting key 0efdf685a8f1c797f752b5f74ad27851
I20260929 03:10:04.400266 30372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } attempt: 3
I20260929 03:10:04.400768 30447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:10:04.407219 30447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.209.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.124@KRBTEST.COM" created.
I20260929 03:10:04.415016 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.400953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:34031 (local address 127.0.0.1:58806)
0929 03:10:04.401125 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.401145 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:04.401275 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:04.404375 (+ 3100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:04.404380 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:04.404635 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:04.404815 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.404825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.406423 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.406427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.407036 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.407048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.407876 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.407880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.407989 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.408644 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:04.408657 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:04.410316 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:04.412774 (+ 2458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.412777 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.412780 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.413054 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.413434 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.413438 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.413440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.413555 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.414375 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:04.414379 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:04.414494 (+ 115us) client_negotiation.cc:769] Sending connection context
0929 03:10:04.414614 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 03:10:04.414816 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":3}
I20260929 03:10:04.416416 30450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.402168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58806 (local address 127.7.209.125:34031)
0929 03:10:04.402665 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.402683 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.402727 (+ 44us) server_negotiation.cc:407] Connection header received
0929 03:10:04.403554 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.403576 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.403999 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.404405 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.404956 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.406278 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.407158 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.407705 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.408329 (+ 624us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.410570 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.410577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.410594 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.410627 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.412668 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.413160 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.413165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.413170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.413343 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.413674 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.413678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.413680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.414250 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.414404 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.414801 (+ 397us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.415178 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":157,"threads_started":1}
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.125@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:10:04.418859 30372 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:10:04.419714 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.419406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40552)
0929 03:10:04.419552 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.419617 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 03:10:04.420538 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.420235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40558)
0929 03:10:04.420417 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.420470 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260929 03:10:04.420795 30372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:45741: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
I20260929 03:10:04.421048 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.409937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40612 (local address 127.7.209.126:40515)
0929 03:10:04.410084 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.410088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.410622 (+ 534us) server_negotiation.cc:407] Connection header received
0929 03:10:04.410884 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.410888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.410946 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.411001 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.411836 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.412641 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.413849 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.414889 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.414986 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.417633 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.417638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.417640 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.417674 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.419168 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.419650 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.419655 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.419657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.419736 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.420028 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.420032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.420033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.420230 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.420299 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.420796 (+ 497us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.420911 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":63}
I20260929 03:10:04.421108 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.409904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:40515 (local address 127.0.0.1:40612)
0929 03:10:04.410469 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.410529 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:04.410741 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:04.411190 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:04.411196 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:04.411569 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:04.411709 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.411715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.412774 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.412778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.413742 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.413754 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.415116 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.415120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.415236 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.415786 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:04.415800 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:04.417471 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:04.419275 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.419284 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.419290 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.419562 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.419838 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.419841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.419843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.419929 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.420387 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:04.420393 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:04.420521 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 03:10:04.420703 (+ 182us) client_negotiation.cc:240] Negotiation successful
0929 03:10:04.420900 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":204,"threads_started":1}
I20260929 03:10:04.422292 30447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.125@KRBTEST.COM for kudu/127.7.209.125@KRBTEST.COM
I20260929 03:10:04.429764 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.422786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:34031 (local address 127.0.0.1:58812)
0929 03:10:04.422876 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.422889 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:04.422963 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:04.423154 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:04.423156 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:04.423309 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:04.423395 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.423400 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.424092 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.424095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.424612 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.424618 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.425018 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.425021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.425105 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.425564 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:04.425577 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:04.426845 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:04.428372 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.428375 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.428377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.428546 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.428797 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.428800 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.428802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.428880 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.429204 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:04.429207 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:04.429290 (+ 83us) client_negotiation.cc:769] Sending connection context
0929 03:10:04.429388 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 03:10:04.429527 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260929 03:10:04.429764 30450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.422874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58812 (local address 127.7.209.125:34031)
0929 03:10:04.422956 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.422960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.422972 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:10:04.423037 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.423040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.423078 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.423136 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.423490 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.424002 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.424703 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.424930 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.425012 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.426958 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.426962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.426965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.426987 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.428284 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.428623 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.428626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.428628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.428727 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.428951 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.428954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.428956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.429117 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.429218 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.429429 (+ 211us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.429579 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":30}
I20260929 03:10:04.430625 30447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.124.keytab.
Entry for principal kudu/127.7.209.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.124.keytab.
I20260929 03:10:04.431388 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.431055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40570)
0929 03:10:04.431231 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.431319 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260929 03:10:04.432065 30398 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
I20260929 03:10:04.432360 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.124:45741
--webserver_interface=127.7.209.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.124.keytab
--principal=kudu/127.7.209.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:40515,127.7.209.125:34031,127.7.209.124:45741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:10:04.433044 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.432854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40578)
0929 03:10:04.432951 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.432993 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 03:10:04.433184 30447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:45741: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
I20260929 03:10:04.451977 30372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } attempt: 1
I20260929 03:10:04.453049 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.452667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40582)
0929 03:10:04.452861 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.452939 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260929 03:10:04.453898 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.453669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40594)
0929 03:10:04.453778 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.453828 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260929 03:10:04.454032 30372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:45741: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
I20260929 03:10:04.482514 30447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } attempt: 1
I20260929 03:10:04.483693 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.483288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40604)
0929 03:10:04.483481 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.483571 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 03:10:04.484493 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.484253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40608)
0929 03:10:04.484358 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.484414 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 03:10:04.484623 30447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:45741: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
I20260929 03:10:04.529198 30372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } attempt: 2
I20260929 03:10:04.530351 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.529987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40612)
0929 03:10:04.530170 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.530249 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 03:10:04.531112 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.530892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40614)
0929 03:10:04.530995 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.531045 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 03:10:04.531237 30372 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:45741: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
I20260929 03:10:04.552803 30447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } attempt: 2
I20260929 03:10:04.553886 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.553459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40620)
0929 03:10:04.553677 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.553767 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260929 03:10:04.554726 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.554473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40622)
0929 03:10:04.554586 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.554646 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 03:10:04.554869 30447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:45741: Network error: Client connection negotiation failed: client connection to 127.7.209.124:45741: connect: Connection refused (error 111)
W20260929 03:10:04.560387 30454 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 03:10:04.560698 30454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:10:04.560746 30454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:10:04.560809 30454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:10:04.565068 30454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:10:04.565137 30454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:10:04.565157 30454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:10:04.565191 30454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:10:04.565217 30454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:10:04.570998 30454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:40515,127.7.209.125:34031,127.7.209.124:45741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.124.keytab
--principal=kudu/127.7.209.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.124:45741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.209.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:10:04.572332 30454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:10:04.573578 30454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:10:04.582772 30460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:04.582731 30459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:04.582774 30462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:10:04.582952 30454 server_base.cc:1034] running on GCE node
I20260929 03:10:04.583633 30454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:10:04.584802 30454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:10:04.585951 30454 hybrid_clock.cc:648] HybridClock initialized: now 1790651404585926 us; error 41 us; skew 500 ppm
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:10:04.588757 30454 init.cc:377] Logged in from keytab as kudu/127.7.209.124@KRBTEST.COM (short username kudu)
I20260929 03:10:04.589773 30454 webserver.cc:460] Webserver started at http://127.7.209.124:38279/ using document root <none> and password file <none>
I20260929 03:10:04.590363 30454 fs_manager.cc:359] Metadata directory not provided
I20260929 03:10:04.590428 30454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:10:04.590642 30454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:10:04.592823 30454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/data/instance:
uuid: "66fcec2f419b4555b6d983e95f59b401"
format_stamp: "Formatted at 2026-09-29 03:10:04 on dist-test-slave-bdps"
server_key: "759d7838360d43b3313d7b5f5d5bcd16"
server_key_iv: "0c06824757a017e5843d312abef9f9e0"
server_key_version: "encryptionkey@0"
I20260929 03:10:04.593398 30454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/wal/instance:
uuid: "66fcec2f419b4555b6d983e95f59b401"
format_stamp: "Formatted at 2026-09-29 03:10:04 on dist-test-slave-bdps"
server_key: "759d7838360d43b3313d7b5f5d5bcd16"
server_key_iv: "0c06824757a017e5843d312abef9f9e0"
server_key_version: "encryptionkey@0"
I20260929 03:10:04.597290 30454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260929 03:10:04.599897 30469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:10:04.601217 30454 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 03:10:04.601336 30454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/wal
uuid: "66fcec2f419b4555b6d983e95f59b401"
format_stamp: "Formatted at 2026-09-29 03:10:04 on dist-test-slave-bdps"
server_key: "759d7838360d43b3313d7b5f5d5bcd16"
server_key_iv: "0c06824757a017e5843d312abef9f9e0"
server_key_version: "encryptionkey@0"
I20260929 03:10:04.601430 30454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:10:04.614842 30454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:10:04.618239 30454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:10:04.618427 30454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:10:04.626012 30454 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.124:45741
I20260929 03:10:04.626327 30521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.124:45741 every 8 connection(s)
I20260929 03:10:04.627149 30454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
I20260929 03:10:04.629911 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 30454
I20260929 03:10:04.630024 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790651120485886-8005-0/minicluster-data/master-2/wal/instance
I20260929 03:10:04.630393 8005 external_mini_cluster.cc:1428] Setting key 5fb752121c2769991b1751757771e73c
I20260929 03:10:04.630409 30522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:10:04.630390 30372 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } attempt: 3
I20260929 03:10:04.636081 30522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.124@KRBTEST.COM
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.124@KRBTEST.COM
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.124@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:10:04.646443 30447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } attempt: 3
I20260929 03:10:04.646894 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.630971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40632)
0929 03:10:04.631546 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.631563 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:04.631663 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:04.634181 (+ 2518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:04.634185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:04.634413 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:04.634547 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.634556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.636430 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.636435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.637266 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.637281 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.638213 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.638217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.638315 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.639027 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:04.639042 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:04.640601 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:04.643220 (+ 2619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.643223 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.643226 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.643436 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.643805 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.643808 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.643810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.643927 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.645014 (+ 1087us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:04.645017 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:04.645117 (+ 100us) client_negotiation.cc:769] Sending connection context
0929 03:10:04.645232 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 03:10:04.645535 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470}
I20260929 03:10:04.646883 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.632286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40646 (local address 127.7.209.124:45741)
0929 03:10:04.632561 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.632570 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.632585 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:10:04.633419 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.633428 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.633879 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.634689 (+ 810us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.635132 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.636257 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.637237 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.638066 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.638615 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.641381 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.641385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.641387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.641412 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.643531 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.644058 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.644061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.644062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.644146 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.644443 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.644447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.644448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.644629 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.644804 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.646138 (+ 1334us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.646255 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":202,"spinlock_wait_cycles":3200,"thread_start_us":103,"threads_started":1}
I20260929 03:10:04.646886 30525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.631867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40632 (local address 127.7.209.124:45741)
0929 03:10:04.632274 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.632295 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.632343 (+ 48us) server_negotiation.cc:407] Connection header received
0929 03:10:04.633404 (+ 1061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.633424 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.633878 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.634231 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.634704 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.636257 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.637397 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.638066 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.638804 (+ 738us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.640792 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.640800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.640812 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.640844 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.643118 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.643534 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.643540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.643545 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.643658 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.644030 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.644033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.644035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.644611 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.644745 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.645386 (+ 641us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.645745 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":262,"server-negotiator.queue_time_us":259,"spinlock_wait_cycles":13056,"thread_start_us":113,"threads_started":1}
I20260929 03:10:04.648983 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.638085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40624 (local address 127.7.209.126:40515)
0929 03:10:04.638212 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.638216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.638586 (+ 370us) server_negotiation.cc:407] Connection header received
0929 03:10:04.638774 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.638777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.638836 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.638912 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.639672 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.640148 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.641245 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.641550 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.641688 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.644755 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.644758 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.644761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.644783 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.646316 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.647470 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.647474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.647476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.647564 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.647866 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.647870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.647872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.648187 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.648256 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.648705 (+ 449us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.648838 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 03:10:04.648986 30527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.638047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:40515 (local address 127.0.0.1:40624)
0929 03:10:04.638456 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.638507 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:04.638663 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:04.638970 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:04.638975 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:04.639437 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:04.639557 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.639562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.640253 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.640256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.641063 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.641072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.641688 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.641692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.641814 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.642410 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:04.642424 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:04.644621 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:04.647061 (+ 2440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.647069 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.647076 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.647356 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.647685 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.647688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.647690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.647773 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.648323 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:04.648329 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:04.648457 (+ 128us) client_negotiation.cc:769] Sending connection context
0929 03:10:04.648626 (+ 169us) client_negotiation.cc:240] Negotiation successful
0929 03:10:04.648810 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":85,"threads_started":1}
I20260929 03:10:04.650126 30522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.125@KRBTEST.COM for kudu/127.7.209.124@KRBTEST.COM
I20260929 03:10:04.653427 30372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18: Bootstrap starting.
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.124@KRBTEST.COM for kudu/127.7.209.125@KRBTEST.COM
I20260929 03:10:04.656764 30451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.647035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40650)
0929 03:10:04.647425 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.647442 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:04.647532 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:04.648584 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:04.648588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:04.648797 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:04.648942 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.648951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.649993 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.649998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.650695 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.650707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.651231 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.651235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.651321 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.651781 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:04.651798 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:04.653312 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:04.655112 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.655115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.655118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.655374 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.655669 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.655673 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.655675 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.655767 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.656203 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:04.656207 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:04.656321 (+ 114us) client_negotiation.cc:769] Sending connection context
0929 03:10:04.656404 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 03:10:04.656574 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284}
I20260929 03:10:04.657296 30525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.647069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40650 (local address 127.7.209.124:45741)
0929 03:10:04.648358 (+ 1289us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.648363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.648376 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:10:04.648407 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.648410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.648451 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.648527 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.649082 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.649860 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.650850 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.651121 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.651208 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.653457 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.653461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.653464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.653487 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.654992 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.655486 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.655489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.655491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.655570 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.655881 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.655884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.655886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.656095 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.656956 (+ 861us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.657033 (+ 77us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.657139 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1219,"thread_start_us":134,"threads_started":1}
I20260929 03:10:04.659061 30372 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18: Neither blocks nor log segments found. Creating new log.
I20260929 03:10:04.659577 30450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.650942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58822 (local address 127.7.209.125:34031)
0929 03:10:04.651053 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.651058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.651071 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:10:04.651106 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.651109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.651156 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.651229 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.651748 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.652469 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.653245 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.653461 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.653547 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.655546 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.655550 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.655552 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.655576 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.657180 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.658151 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.658154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.658156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.658234 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.658531 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.658534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.658536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.658797 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.658872 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.659308 (+ 436us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.659424 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:10:04.660298 30527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.650690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:34031 (local address 127.0.0.1:58822)
0929 03:10:04.650809 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.650823 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:04.650884 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:04.651349 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:04.651351 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:04.651516 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:04.651633 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.651640 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.652569 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.652573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.653141 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.653150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.653554 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.653558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.653689 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.654101 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:04.654111 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:04.655407 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:04.657801 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.657804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.657806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.658039 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.658345 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.658348 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.658350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.658437 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.658948 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:04.658951 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:04.659040 (+ 89us) client_negotiation.cc:769] Sending connection context
0929 03:10:04.660003 (+ 963us) client_negotiation.cc:240] Negotiation successful
0929 03:10:04.660147 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 03:10:04.660360 30372 log.cc:824] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18: Log is configured to *not* fsync() on all Append() calls
I20260929 03:10:04.660991 30447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a: Bootstrap starting.
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:10:04.661406 30522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:10:04.663620 30372 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18: No bootstrap required, opened a new log
I20260929 03:10:04.664678 30447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a: Neither blocks nor log segments found. Creating new log.
I20260929 03:10:04.665267 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.654390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40640 (local address 127.7.209.126:40515)
0929 03:10:04.654522 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.654526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.655291 (+ 765us) server_negotiation.cc:407] Connection header received
0929 03:10:04.655486 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.655489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.655538 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.655586 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.656589 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.657355 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.658477 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.658797 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.658886 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.661691 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.661695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.661698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.661722 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.663226 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.663742 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.663745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.663747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.663839 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.664147 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.664150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.664152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.664376 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.664537 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.664978 (+ 441us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.665103 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 03:10:04.665719 30447 log.cc:824] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a: Log is configured to *not* fsync() on all Append() calls
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651404, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.124@KRBTEST.COM for kudu/127.7.209.124@KRBTEST.COM
I20260929 03:10:04.668587 30372 raft_consensus.cc:348] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } }
I20260929 03:10:04.668901 30372 raft_consensus.cc:729] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492b4abcab744ca199404e9c0c280f18, State: Initialized, Role: FOLLOWER
I20260929 03:10:04.669338 30372 consensus_queue.cc:260] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } }
I20260929 03:10:04.670674 30447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a: No bootstrap required, opened a new log
I20260929 03:10:04.671200 30372 sys_catalog.cc:564] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:10:04.671473 30527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.661964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:45741 (local address 127.0.0.1:40658)
0929 03:10:04.662192 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:04.662256 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:04.662343 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:04.663358 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:04.663361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:04.663518 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:04.663618 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.663625 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.664583 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.664586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.665328 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:04.665339 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.666044 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.666047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:04.666118 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.666581 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:04.666594 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:04.667896 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:04.669908 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.669911 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.669913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.670123 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.670401 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:04.670403 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:04.670405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:04.670485 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:04.670955 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:04.670958 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:04.671047 (+ 89us) client_negotiation.cc:769] Sending connection context
0929 03:10:04.671149 (+ 102us) client_negotiation.cc:240] Negotiation successful
0929 03:10:04.671284 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172}
I20260929 03:10:04.672355 30525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.662035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.7.209.124:45741)
0929 03:10:04.662497 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.662501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.662512 (+ 11us) server_negotiation.cc:407] Connection header received
0929 03:10:04.662541 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.662544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.662607 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.663704 (+ 1097us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.663806 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.664483 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.665445 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.665843 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.665939 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.668013 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.668017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.668020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.668042 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.669818 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.670210 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.670213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.670215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.670300 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.670565 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.670568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.670569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.670788 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.670878 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.672056 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.672179 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Sep 29 03:10:04 dist-test-slave-bdps krb5kdc[30291](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.125@KRBTEST.COM
I20260929 03:10:04.675438 30447 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } }
I20260929 03:10:04.675709 30447 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbdcf50917f463f855bf0af04da808a, State: Initialized, Role: FOLLOWER
I20260929 03:10:04.676097 30447 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } }
I20260929 03:10:04.676173 30522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401: Bootstrap starting.
I20260929 03:10:04.677762 30447 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:10:04.679593 30522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401: Neither blocks nor log segments found. Creating new log.
I20260929 03:10:04.680689 30522 log.cc:824] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401: Log is configured to *not* fsync() on all Append() calls
I20260929 03:10:04.677850 30304 master_runner.cc:186] Error getting master registration for 127.7.209.125:34031: OK, instance_id { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" instance_seqno: 1790651404393718 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 03:10:04.681543 30304 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 03:10:04.685339 30522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401: No bootstrap required, opened a new log
I20260929 03:10:04.685813 30531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } } }
I20260929 03:10:04.685971 30531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [sys.catalog]: This master's current role is: FOLLOWER
W20260929 03:10:04.686749 30542 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:10:04.686827 30542 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:10:04.687288 30450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.667924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58836 (local address 127.7.209.125:34031)
0929 03:10:04.668045 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.668050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.668224 (+ 174us) server_negotiation.cc:407] Connection header received
0929 03:10:04.668417 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.668420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.668469 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.668520 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.669382 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.670155 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.671194 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.671520 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.671599 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.682382 (+ 10783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.682386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.682390 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.682423 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.684292 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.684971 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.684975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.684977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.685077 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.685562 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.685566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.685568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.685907 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.686289 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.686991 (+ 702us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.687115 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:10:04.686125 30379 master_runner.cc:186] Error getting master registration for 127.7.209.124:45741: OK, instance_id { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" instance_seqno: 1790651404623954 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 03:10:04.689806 30544 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } } }
I20260929 03:10:04.689963 30544 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:04.690544 30522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } }
I20260929 03:10:04.690840 30522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66fcec2f419b4555b6d983e95f59b401, State: Initialized, Role: FOLLOWER
I20260929 03:10:04.691293 30522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } }
I20260929 03:10:04.692898 30522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:10:04.693032 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260929 03:10:04.693832 30555 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:10:04.694216 30555 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:10:04.689806 30379 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 03:10:04.694684 30346 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "492b4abcab744ca199404e9c0c280f18"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40640
I20260929 03:10:04.694953 30346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 03:10:04.695087 30346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:10:04.696707 30346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } }
I20260929 03:10:04.698352 30421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "492b4abcab744ca199404e9c0c280f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bfbdcf50917f463f855bf0af04da808a"
I20260929 03:10:04.698606 30421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:10:04.700064 30421 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492b4abcab744ca199404e9c0c280f18 in term 1.
I20260929 03:10:04.700515 30322 leader_election.cc:302] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492b4abcab744ca199404e9c0c280f18, bfbdcf50917f463f855bf0af04da808a; no voters:
I20260929 03:10:04.700830 30531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:10:04.701116 30531 raft_consensus.cc:686] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [term 1 LEADER]: Becoming Leader. State: Replica: 492b4abcab744ca199404e9c0c280f18, State: Running, Role: LEADER
I20260929 03:10:04.701486 30531 consensus_queue.cc:237] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } }
I20260929 03:10:04.701671 30346 leader_election.cc:288] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [CANDIDATE]: Term 1 election: Requested vote from peers bfbdcf50917f463f855bf0af04da808a (127.7.209.125:34031), 66fcec2f419b4555b6d983e95f59b401 (127.7.209.124:45741)
I20260929 03:10:04.702441 30496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "492b4abcab744ca199404e9c0c280f18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "66fcec2f419b4555b6d983e95f59b401"
I20260929 03:10:04.702678 30496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:10:04.703763 30570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 492b4abcab744ca199404e9c0c280f18. Latest consensus state: current_term: 1 leader_uuid: "492b4abcab744ca199404e9c0c280f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } } }
I20260929 03:10:04.703869 30570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [sys.catalog]: This master's current role is: LEADER
I20260929 03:10:04.704170 30496 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 492b4abcab744ca199404e9c0c280f18 in term 1.
I20260929 03:10:04.705806 30557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } } }
I20260929 03:10:04.705942 30557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:04.706342 30576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:10:04.708243 30454 master_runner.cc:263] No leader master found from master 66fcec2f419b4555b6d983e95f59b401
I20260929 03:10:04.708304 30454 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 03:10:04.708557 30576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260929 03:10:04.713446 30569 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:10:04.713529 30569 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:10:04.723596 30450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.704468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.7.209.125:34031)
0929 03:10:04.704596 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.704602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.705300 (+ 698us) server_negotiation.cc:407] Connection header received
0929 03:10:04.705535 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.705538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.705593 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.705683 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.706835 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.707619 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.708983 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.709320 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.709415 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.711679 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.711683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.711686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.711715 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.713512 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.721115 (+ 7603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.721120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.721122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.721239 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.722529 (+ 1290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.722533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.722535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.722775 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.722887 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.723236 (+ 349us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.723400 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:10:04.731633 30525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.705380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40660 (local address 127.7.209.124:45741)
0929 03:10:04.705568 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.705573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.714196 (+ 8623us) server_negotiation.cc:407] Connection header received
0929 03:10:04.714358 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.714361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.714412 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.714465 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.715663 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.716436 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.717683 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.718031 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.718106 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.726784 (+ 8678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.726788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.726792 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.726824 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.728562 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.729055 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.729059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.729061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.729147 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.730577 (+ 1430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.730581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.730583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.730823 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.730961 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.731297 (+ 336us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.731447 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260929 03:10:04.734354 30496 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [term 1 FOLLOWER]: Refusing update from remote peer 492b4abcab744ca199404e9c0c280f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:10:04.735144 30570 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:10:04.737042 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:04.704844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40654 (local address 127.7.209.126:40515)
0929 03:10:04.704956 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.704960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.713880 (+ 8920us) server_negotiation.cc:407] Connection header received
0929 03:10:04.714003 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.714006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.714067 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.714144 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.718800 (+ 4656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.719645 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.725487 (+ 5842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:04.725896 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:04.725979 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:04.733391 (+ 7412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:04.733395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:04.733399 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:04.733432 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:04.735129 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.735637 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.735641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.735642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.735728 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:04.736017 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:04.736021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:04.736023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:04.736246 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:04.736386 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:04.736753 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 03:10:04.736881 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 03:10:04.739610 30557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 492b4abcab744ca199404e9c0c280f18. Latest consensus state: current_term: 1 leader_uuid: "492b4abcab744ca199404e9c0c280f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } } }
I20260929 03:10:04.739754 30557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:04.740778 30421 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [term 1 FOLLOWER]: Refusing update from remote peer 492b4abcab744ca199404e9c0c280f18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:10:04.741398 30570 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:10:04.745658 30544 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 492b4abcab744ca199404e9c0c280f18. Latest consensus state: current_term: 1 leader_uuid: "492b4abcab744ca199404e9c0c280f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } } }
I20260929 03:10:04.745797 30544 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:04.749017 30570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "492b4abcab744ca199404e9c0c280f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } } }
I20260929 03:10:04.749137 30570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [sys.catalog]: This master's current role is: LEADER
I20260929 03:10:04.750370 30557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "492b4abcab744ca199404e9c0c280f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } } }
I20260929 03:10:04.750478 30557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:04.750855 30582 mvcc.cc:205] Tried to move back new op lower bound from 7334508153782226944 to 7334508153661317120. Current Snapshot: MvccSnapshot[applied={T|T < 7334508153782226944}]
I20260929 03:10:04.751437 30531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "492b4abcab744ca199404e9c0c280f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } } }
I20260929 03:10:04.751554 30531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18 [sys.catalog]: This master's current role is: LEADER
I20260929 03:10:04.752363 30544 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "492b4abcab744ca199404e9c0c280f18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "492b4abcab744ca199404e9c0c280f18" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 40515 } } peers { permanent_uuid: "bfbdcf50917f463f855bf0af04da808a" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 34031 } } peers { permanent_uuid: "66fcec2f419b4555b6d983e95f59b401" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 45741 } } }
I20260929 03:10:04.752470 30544 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:04.757062 30584 mvcc.cc:205] Tried to move back new op lower bound from 7334508153782226944 to 7334508153661317120. Current Snapshot: MvccSnapshot[applied={T|T < 7334508153782226944}]
I20260929 03:10:04.758378 30576 catalog_manager.cc:1347] Generated new cluster ID: 9fa9a81ccae64fb7b6691c90800d26cd
I20260929 03:10:04.758442 30576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:10:04.788882 30576 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:10:04.790643 30576 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:10:04.812873 30576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 492b4abcab744ca199404e9c0c280f18: Generated new TSK 0
I20260929 03:10:04.814303 30576 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 03:10:04.850816 30375 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:04.846766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40660 (local address 127.7.209.126:40515)
0929 03:10:04.846944 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.846949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.849214 (+ 2265us) server_negotiation.cc:407] Connection header received
0929 03:10:04.849478 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.849482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.849545 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.849603 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.850637 (+ 1034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40660: BlockingRecv error: recv got EOF from 127.0.0.1:40660 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260929 03:10:04.852972 30525 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:04.847730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.7.209.124:45741)
0929 03:10:04.847868 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.847872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.849945 (+ 2073us) server_negotiation.cc:407] Connection header received
0929 03:10:04.851795 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.851799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.851861 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.851920 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.852823 (+ 903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40662: BlockingRecv error: recv got EOF from 127.0.0.1:40662 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260929 03:10:04.854594 30450 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:04.848503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58854 (local address 127.7.209.125:34031)
0929 03:10:04.848646 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.848650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.851492 (+ 2842us) server_negotiation.cc:407] Connection header received
0929 03:10:04.852075 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.852079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.852139 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.852193 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.854460 (+ 2267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58854: BlockingRecv error: recv got EOF from 127.0.0.1:58854 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260929 03:10:04.860428 30450 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:04.858445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58862 (local address 127.7.209.125:34031)
0929 03:10:04.858587 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.858591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.859229 (+ 638us) server_negotiation.cc:407] Connection header received
0929 03:10:04.859504 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.859508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.859605 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.859672 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.860298 (+ 626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58862: BlockingRecv error: recv got EOF from 127.0.0.1:58862 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260929 03:10:04.866616 30450 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:04.864824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58874 (local address 127.7.209.125:34031)
0929 03:10:04.864943 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.864947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.865526 (+ 579us) server_negotiation.cc:407] Connection header received
0929 03:10:04.865812 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.865816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.865875 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.865931 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.866498 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58874: BlockingRecv error: recv got EOF from 127.0.0.1:58874 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260929 03:10:04.874054 30450 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:04.872312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58876 (local address 127.7.209.125:34031)
0929 03:10:04.872424 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.872429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.872978 (+ 549us) server_negotiation.cc:407] Connection header received
0929 03:10:04.873180 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.873183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.873253 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.873312 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.873937 (+ 625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58876: BlockingRecv error: recv got EOF from 127.0.0.1:58876 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260929 03:10:04.881706 30450 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:04.880180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58892 (local address 127.7.209.125:34031)
0929 03:10:04.880321 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.880325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.880813 (+ 488us) server_negotiation.cc:407] Connection header received
0929 03:10:04.880974 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.880977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.881027 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.881075 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.881573 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58892: BlockingRecv error: recv got EOF from 127.0.0.1:58892 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260929 03:10:04.894780 30450 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:04.892838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.7.209.125:34031)
0929 03:10:04.892975 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.892981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.893645 (+ 664us) server_negotiation.cc:407] Connection header received
0929 03:10:04.893872 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.893876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.893935 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.893990 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.894627 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58908: BlockingRecv error: recv got EOF from 127.0.0.1:58908 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260929 03:10:04.914639 30450 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:04.913318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58922 (local address 127.7.209.125:34031)
0929 03:10:04.913460 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.913465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.913726 (+ 261us) server_negotiation.cc:407] Connection header received
0929 03:10:04.913869 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.913872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.913941 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.914051 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.914487 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58922: BlockingRecv error: recv got EOF from 127.0.0.1:58922 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260929 03:10:04.949992 30450 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:04.948814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58934 (local address 127.7.209.125:34031)
0929 03:10:04.949004 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:04.949008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:04.949103 (+ 95us) server_negotiation.cc:407] Connection header received
0929 03:10:04.949242 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:04.949245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:04.949301 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:04.949392 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:04.949834 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58934: BlockingRecv error: recv got EOF from 127.0.0.1:58934 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260929 03:10:05.018004 30450 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:05.016565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58942 (local address 127.7.209.125:34031)
0929 03:10:05.016761 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:05.016767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:05.016948 (+ 181us) server_negotiation.cc:407] Connection header received
0929 03:10:05.017143 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:05.017147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:05.017208 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:05.017288 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:05.017851 (+ 563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58942: BlockingRecv error: recv got EOF from 127.0.0.1:58942 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260929 03:10:05.149883 30450 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:05.148451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58950 (local address 127.7.209.125:34031)
0929 03:10:05.148600 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:05.148605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:05.148809 (+ 204us) server_negotiation.cc:407] Connection header received
0929 03:10:05.148986 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:05.148990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:05.149052 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:05.149133 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:05.149720 (+ 587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58950: BlockingRecv error: recv got EOF from 127.0.0.1:58950 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260929 03:10:05.409849 30450 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:05.408376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.7.209.125:34031)
0929 03:10:05.408542 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:05.408548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:05.408736 (+ 188us) server_negotiation.cc:407] Connection header received
0929 03:10:05.408920 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:05.408923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:05.408985 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:05.409072 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:05.409710 (+ 638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58962: BlockingRecv error: recv got EOF from 127.0.0.1:58962 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 03:10:05.696138 30555 catalog_manager.cc:1259] Loaded cluster ID: 9fa9a81ccae64fb7b6691c90800d26cd
I20260929 03:10:05.696272 30555 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a: loading cluster ID for follower catalog manager: success
I20260929 03:10:05.698311 30555 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a: acquiring CA information for follower catalog manager: success
I20260929 03:10:05.699734 30555 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bfbdcf50917f463f855bf0af04da808a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 03:10:05.715029 30569 catalog_manager.cc:1259] Loaded cluster ID: 9fa9a81ccae64fb7b6691c90800d26cd
I20260929 03:10:05.715121 30569 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401: loading cluster ID for follower catalog manager: success
I20260929 03:10:05.717032 30569 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401: acquiring CA information for follower catalog manager: success
I20260929 03:10:05.718406 30569 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 66fcec2f419b4555b6d983e95f59b401: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 03:10:05.929047 30653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:05.924417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58966 (local address 127.7.209.125:34031)
0929 03:10:05.924701 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:05.924706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:05.924820 (+ 114us) server_negotiation.cc:407] Connection header received
0929 03:10:05.925030 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:05.925033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:05.925202 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:05.925325 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:10:05.925688 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:05.926291 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:05.927266 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:05.927511 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:05.927605 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:05.927996 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:10:05.928608 (+ 612us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:10:05.928747 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:05.928781 (+ 34us) server_negotiation.cc:299] Negotiation successful
0929 03:10:05.928856 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":102,"threads_started":1}
I20260929 03:10:05.936822 30659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:05.932763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40676 (local address 127.7.209.124:45741)
0929 03:10:05.933039 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:05.933045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:05.933061 (+ 16us) server_negotiation.cc:407] Connection header received
0929 03:10:05.933171 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:05.933174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:05.933336 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:05.933457 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:10:05.933701 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:05.934270 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:05.935114 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:05.935409 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:05.935495 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:05.935811 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:10:05.936370 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:10:05.936490 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:05.936577 (+ 87us) server_negotiation.cc:299] Negotiation successful
0929 03:10:05.936636 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260929 03:10:05.943527 30653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:05.939971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58976 (local address 127.7.209.125:34031)
0929 03:10:05.940105 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:05.940110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:05.940123 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:10:05.940178 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:05.940180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:05.940239 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:05.940315 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:10:05.940606 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:05.941361 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:05.942442 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:05.942667 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:05.942753 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:05.943005 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:10:05.943097 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:10:05.943171 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:05.943307 (+ 136us) server_negotiation.cc:299] Negotiation successful
0929 03:10:05.943372 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 03:10:05.943938 30659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:05.940187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.7.209.124:45741)
0929 03:10:05.940294 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:05.940297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:05.940533 (+ 236us) server_negotiation.cc:407] Connection header received
0929 03:10:05.940676 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:05.940679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:05.940726 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:05.940798 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:10:05.941011 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:05.941724 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:05.942914 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:05.943171 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:05.943269 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:05.943483 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:10:05.943558 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:10:05.943617 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:05.943770 (+ 153us) server_negotiation.cc:299] Negotiation successful
0929 03:10:05.943818 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 03:10:05.944753 30666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:05.939741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.7.209.126:40515)
0929 03:10:05.939983 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:05.939988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:05.940376 (+ 388us) server_negotiation.cc:407] Connection header received
0929 03:10:05.940506 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:05.940509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:05.940702 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:05.940817 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:10:05.941248 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:05.942065 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:05.943149 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:05.943468 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:05.943564 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:05.943906 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:10:05.944316 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:10:05.944399 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:05.944535 (+ 136us) server_negotiation.cc:299] Negotiation successful
0929 03:10:05.944606 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":71,"threads_started":1}
I20260929 03:10:05.951437 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 30304
I20260929 03:10:05.958220 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 30379
I20260929 03:10:05.964112 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 30454
2026-09-29T03:10:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3080 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 03:10:06 dist-test-slave-bdps krb5kdc[30670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 03:10:06 dist-test-slave-bdps krb5kdc[30670](info): set up 2 sockets
Sep 29 03:10:06 dist-test-slave-bdps krb5kdc[30670](info): commencing operation
krb5kdc: starting...
W20260929 03:10:06.926273 8005 mini_kdc.cc:121] Time spent starting KDC: real 0.940s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/test-user.keytab.
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T03:10:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T03:10:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.209.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
Entry for principal kudu/127.7.209.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab.
I20260929 03:10:07.052130 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:38311
--webserver_interface=127.7.209.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:38311,127.7.209.125:42823,127.7.209.124:40855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 03:10:07.177420 30683 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 03:10:07.177824 30683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:10:07.177865 30683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:10:07.177930 30683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:10:07.182015 30683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:10:07.182075 30683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:10:07.182104 30683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:10:07.182125 30683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:10:07.182147 30683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:10:07.187638 30683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:38311,127.7.209.125:42823,127.7.209.124:40855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.126.keytab
--principal=kudu/127.7.209.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.126:38311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.209.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:10:07.188932 30683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:10:07.190161 30683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:10:07.199426 30688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:07.199463 30691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:07.199456 30689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:10:07.199860 30683 server_base.cc:1034] running on GCE node
I20260929 03:10:07.200444 30683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:10:07.201725 30683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:10:07.202878 30683 hybrid_clock.cc:648] HybridClock initialized: now 1790651407202853 us; error 44 us; skew 500 ppm
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:10:07.205878 30683 init.cc:377] Logged in from keytab as kudu/127.7.209.126@KRBTEST.COM (short username kudu)
I20260929 03:10:07.206943 30683 webserver.cc:460] Webserver started at http://127.7.209.126:39411/ using document root <none> and password file <none>
I20260929 03:10:07.207604 30683 fs_manager.cc:359] Metadata directory not provided
I20260929 03:10:07.207692 30683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:10:07.207929 30683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:10:07.210433 30683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/data/instance:
uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207"
format_stamp: "Formatted at 2026-09-29 03:10:07 on dist-test-slave-bdps"
server_key: "1cd95cfc3144389166aedea0e024012b"
server_key_iv: "8e047a95cd69d4153ada42889704bf4a"
server_key_version: "encryptionkey@0"
I20260929 03:10:07.211051 30683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/wal/instance:
uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207"
format_stamp: "Formatted at 2026-09-29 03:10:07 on dist-test-slave-bdps"
server_key: "1cd95cfc3144389166aedea0e024012b"
server_key_iv: "8e047a95cd69d4153ada42889704bf4a"
server_key_version: "encryptionkey@0"
I20260929 03:10:07.215266 30683 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:10:07.217924 30698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:10:07.219342 30683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 03:10:07.219465 30683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/wal
uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207"
format_stamp: "Formatted at 2026-09-29 03:10:07 on dist-test-slave-bdps"
server_key: "1cd95cfc3144389166aedea0e024012b"
server_key_iv: "8e047a95cd69d4153ada42889704bf4a"
server_key_version: "encryptionkey@0"
I20260929 03:10:07.219563 30683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:10:07.236054 30683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:10:07.239097 30683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:10:07.239284 30683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:10:07.247053 30683 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.126:38311
I20260929 03:10:07.247350 30750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.126:38311 every 8 connection(s)
I20260929 03:10:07.248277 30683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/data/info.pb
I20260929 03:10:07.249389 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 30683
I20260929 03:10:07.249517 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-0/wal/instance
I20260929 03:10:07.249878 8005 external_mini_cluster.cc:1428] Setting key 36f376d61b6e12bb4c84f48aca0e2b01
I20260929 03:10:07.251870 30751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:10:07.256332 30751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.209.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.125@KRBTEST.COM" created.
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:10:07.272516 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.259120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55042 (local address 127.7.209.126:38311)
0929 03:10:07.259581 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.259602 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.259949 (+ 347us) server_negotiation.cc:407] Connection header received
0929 03:10:07.261012 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.261017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.261258 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.261446 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.262300 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.263499 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.264740 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.265309 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.265486 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.268020 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.268027 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.268041 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.268062 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.269704 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.270154 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.270160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.270163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.270280 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.270521 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.270523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.270524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.270917 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.271023 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.271378 (+ 355us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.271607 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":302,"thread_start_us":114,"threads_started":1}
I20260929 03:10:07.272516 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.259120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38311 (local address 127.0.0.1:55042)
0929 03:10:07.259785 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.259828 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:07.260701 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:07.261476 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:07.261481 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:07.261885 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:07.262158 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.262170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.263619 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.263623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.264619 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.264629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.265443 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.265446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.265555 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.266193 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:07.266208 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:07.267874 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:07.269798 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.269806 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.269814 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.270029 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.270349 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.270351 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.270352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.270455 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.270998 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:07.271003 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:07.271122 (+ 119us) client_negotiation.cc:769] Sending connection context
0929 03:10:07.271325 (+ 203us) client_negotiation.cc:240] Negotiation successful
0929 03:10:07.271661 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":62,"spinlock_wait_cycles":27520,"thread_start_us":190,"threads_started":1}
I20260929 03:10:07.274955 30751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:10:07.275645 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.275394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:42823 (local address 127.0.0.1:59464)
0929 03:10:07.275490 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.275560 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:42823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.125.keytab.
Entry for principal kudu/127.7.209.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.125.keytab.
W20260929 03:10:07.275913 30702 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.209.125:42823: connect: Connection refused (error 111)
I20260929 03:10:07.276696 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.276534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:42823 (local address 127.0.0.1:59466)
0929 03:10:07.276608 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.276649 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:42823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260929 03:10:07.276823 30751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.125:42823: Network error: Client connection negotiation failed: client connection to 127.7.209.125:42823: connect: Connection refused (error 111)
I20260929 03:10:07.276872 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.125:42823
--webserver_interface=127.7.209.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.125.keytab
--principal=kudu/127.7.209.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:38311,127.7.209.125:42823,127.7.209.124:40855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:10:07.326081 30751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } attempt: 1
I20260929 03:10:07.327244 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.326831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:42823 (local address 127.0.0.1:33990)
0929 03:10:07.327022 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.327113 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:42823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260929 03:10:07.328042 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.327820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:42823 (local address 127.0.0.1:33992)
0929 03:10:07.327917 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.327971 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:42823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 03:10:07.328177 30751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.125:42823: Network error: Client connection negotiation failed: client connection to 127.7.209.125:42823: connect: Connection refused (error 111)
I20260929 03:10:07.396390 30751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } attempt: 2
I20260929 03:10:07.397476 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.397060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:42823 (local address 127.0.0.1:34006)
0929 03:10:07.397281 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.397367 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:42823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 03:10:07.398372 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.398085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:42823 (local address 127.0.0.1:34016)
0929 03:10:07.398247 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.398299 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.125:42823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260929 03:10:07.398519 30751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.125:42823: Network error: Client connection negotiation failed: client connection to 127.7.209.125:42823: connect: Connection refused (error 111)
W20260929 03:10:07.403005 30758 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 03:10:07.403302 30758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:10:07.403331 30758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:10:07.403370 30758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:10:07.407510 30758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:10:07.407584 30758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:10:07.407603 30758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:10:07.407619 30758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:10:07.407634 30758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:10:07.413676 30758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:38311,127.7.209.125:42823,127.7.209.124:40855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.125.keytab
--principal=kudu/127.7.209.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.125:42823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.209.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:10:07.415175 30758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:10:07.416529 30758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:10:07.426476 30764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:07.426436 30763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:07.426494 30766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:10:07.426803 30758 server_base.cc:1034] running on GCE node
I20260929 03:10:07.427412 30758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:10:07.428719 30758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:10:07.429864 30758 hybrid_clock.cc:648] HybridClock initialized: now 1790651407429838 us; error 41 us; skew 500 ppm
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:10:07.432770 30758 init.cc:377] Logged in from keytab as kudu/127.7.209.125@KRBTEST.COM (short username kudu)
I20260929 03:10:07.433789 30758 webserver.cc:460] Webserver started at http://127.7.209.125:36369/ using document root <none> and password file <none>
I20260929 03:10:07.434451 30758 fs_manager.cc:359] Metadata directory not provided
I20260929 03:10:07.434521 30758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:10:07.434746 30758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:10:07.437059 30758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/data/instance:
uuid: "4db5252cf9b64eb39e6334f545d2d41f"
format_stamp: "Formatted at 2026-09-29 03:10:07 on dist-test-slave-bdps"
server_key: "7b148bac550fed09ede7a135a9c3f39e"
server_key_iv: "86c0c073671123c5ed43a94e210837ca"
server_key_version: "encryptionkey@0"
I20260929 03:10:07.437707 30758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/wal/instance:
uuid: "4db5252cf9b64eb39e6334f545d2d41f"
format_stamp: "Formatted at 2026-09-29 03:10:07 on dist-test-slave-bdps"
server_key: "7b148bac550fed09ede7a135a9c3f39e"
server_key_iv: "86c0c073671123c5ed43a94e210837ca"
server_key_version: "encryptionkey@0"
I20260929 03:10:07.441857 30758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 03:10:07.444475 30773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:10:07.445981 30758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 03:10:07.446105 30758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/wal
uuid: "4db5252cf9b64eb39e6334f545d2d41f"
format_stamp: "Formatted at 2026-09-29 03:10:07 on dist-test-slave-bdps"
server_key: "7b148bac550fed09ede7a135a9c3f39e"
server_key_iv: "86c0c073671123c5ed43a94e210837ca"
server_key_version: "encryptionkey@0"
I20260929 03:10:07.446223 30758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:10:07.468933 30758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:10:07.472586 30758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:10:07.472810 30758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:10:07.481011 30758 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.125:42823
I20260929 03:10:07.481312 30825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.125:42823 every 8 connection(s)
I20260929 03:10:07.482187 30758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/data/info.pb
I20260929 03:10:07.485131 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 30758
I20260929 03:10:07.485282 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-1/wal/instance
I20260929 03:10:07.485687 30826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:10:07.485729 8005 external_mini_cluster.cc:1428] Setting key 513ea1867f25c723c7cd8b1f83e9d9b4
I20260929 03:10:07.489661 30751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } attempt: 3
I20260929 03:10:07.491245 30826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.209.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.209.124@KRBTEST.COM" created.
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.125@KRBTEST.COM
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.125@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:10:07.504979 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.490422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:42823 (local address 127.0.0.1:34032)
0929 03:10:07.490714 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.490733 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:07.490859 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:07.494115 (+ 3256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:07.494120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:07.494361 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:07.494520 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.494528 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.496756 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.496760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.497494 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.497504 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.498200 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.498202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.498290 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.498901 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:07.498915 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:07.500383 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:07.502843 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.502846 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.502849 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.503038 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.503413 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.503416 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.503418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.503513 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.504398 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:07.504401 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:07.504508 (+ 107us) client_negotiation.cc:769] Sending connection context
0929 03:10:07.504606 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 03:10:07.504767 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":24}
I20260929 03:10:07.505316 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.493393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.7.209.126:38311)
0929 03:10:07.493532 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.493537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.493973 (+ 436us) server_negotiation.cc:407] Connection header received
0929 03:10:07.494162 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.494165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.494234 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.494284 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.495491 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.496033 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.497273 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.497690 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.497800 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.501190 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.501194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.501196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.501224 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.503430 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.503938 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.503941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.503943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.504020 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.504354 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.504357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.504359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.504555 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.504643 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.505046 (+ 403us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.505158 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":57}
I20260929 03:10:07.506424 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.491386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.7.209.125:42823)
0929 03:10:07.491984 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.492008 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.492069 (+ 61us) server_negotiation.cc:407] Connection header received
0929 03:10:07.493088 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.493121 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.493762 (+ 641us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.494727 (+ 965us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.495215 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.496614 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.497598 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.498095 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.498197 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.500538 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.500546 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.500563 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.500592 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.502737 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.503141 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.503148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.503156 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.503300 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.503623 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.503626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.503628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.504260 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.504432 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.504803 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.505159 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":398,"spinlock_wait_cycles":896,"thread_start_us":166,"threads_started":1}
I20260929 03:10:07.506422 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.493341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38311 (local address 127.0.0.1:51738)
0929 03:10:07.493817 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.493870 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:07.494064 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:07.494509 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:07.494517 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:07.494876 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:07.495355 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.495366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.496138 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.496140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.497111 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.497122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.497849 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.497853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.498046 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.498752 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:07.498773 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:07.501005 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:07.503547 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.503557 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.503564 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.503830 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.504119 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.504122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.504124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.504201 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.504689 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:07.504696 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:07.504812 (+ 116us) client_negotiation.cc:769] Sending connection context
0929 03:10:07.505500 (+ 688us) client_negotiation.cc:240] Negotiation successful
0929 03:10:07.505712 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":2,"thread_start_us":147,"threads_started":1}
I20260929 03:10:07.508047 30826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:10:07.508947 30751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:10:07.509723 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.509421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47176)
0929 03:10:07.509574 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.509652 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260929 03:10:07.510457 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.510202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47186)
0929 03:10:07.510346 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.510397 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260929 03:10:07.510635 30751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:40855: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.125@KRBTEST.COM for kudu/127.7.209.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.209.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.124.keytab.
Entry for principal kudu/127.7.209.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.124.keytab.
I20260929 03:10:07.516018 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.508630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:42823 (local address 127.0.0.1:34040)
0929 03:10:07.508741 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.508753 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:07.508820 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:07.509331 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:07.509333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:07.509486 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:07.509572 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.509576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.510502 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.510505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.511306 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.511315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.511741 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.511745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.511848 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.512247 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:07.512258 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:07.513391 (+ 1133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:07.514853 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.514857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.514859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.515054 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.515277 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.515280 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.515281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.515348 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.515649 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:07.515651 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:07.515719 (+ 68us) client_negotiation.cc:769] Sending connection context
0929 03:10:07.515810 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 03:10:07.515905 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 03:10:07.516420 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.508999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.7.209.125:42823)
0929 03:10:07.509148 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.509152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.509164 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:10:07.509194 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.509197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.509249 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.509318 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.509710 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.510410 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.511416 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.511646 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.511737 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.513524 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.513527 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.513530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.513553 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.514748 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.515127 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.515129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.515130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.515194 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.515413 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.515415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.515416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.515568 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.515666 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.516194 (+ 528us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.516303 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 03:10:07.516757 8005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
/tmp/dist-test-taskC1pyL7/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.124:40855
--webserver_interface=127.7.209.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.124.keytab
--principal=kudu/127.7.209.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.209.84:39575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.209.126:38311,127.7.209.125:42823,127.7.209.124:40855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 03:10:07.516985 30826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:10:07.517663 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.517370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47202)
0929 03:10:07.517475 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.517555 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 03:10:07.517876 30774 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
I20260929 03:10:07.518705 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.518540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47212)
0929 03:10:07.518621 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.518660 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 03:10:07.518828 30826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:40855: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
I20260929 03:10:07.541795 30751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } attempt: 1
I20260929 03:10:07.542878 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.542520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47222)
0929 03:10:07.542695 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.542769 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 03:10:07.543680 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.543463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47230)
0929 03:10:07.543566 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.543615 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 03:10:07.543821 30751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:40855: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
I20260929 03:10:07.568142 30826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } attempt: 1
I20260929 03:10:07.569341 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.568936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47242)
0929 03:10:07.569125 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.569216 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 03:10:07.570186 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.569960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47244)
0929 03:10:07.570062 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.570114 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 03:10:07.570353 30826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:40855: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
I20260929 03:10:07.618988 30751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } attempt: 2
I20260929 03:10:07.620111 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.619661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47256)
0929 03:10:07.619886 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.619983 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260929 03:10:07.620963 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.620712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47266)
0929 03:10:07.620834 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.620896 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260929 03:10:07.621109 30751 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:40855: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
I20260929 03:10:07.638517 30826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } attempt: 2
I20260929 03:10:07.639309 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.639002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47270)
0929 03:10:07.639150 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.639214 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 03:10:07.640002 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.639793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47280)
0929 03:10:07.639886 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.639937 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260929 03:10:07.640115 30826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.209.124:40855: Network error: Client connection negotiation failed: client connection to 127.7.209.124:40855: connect: Connection refused (error 111)
W20260929 03:10:07.643256 30832 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 03:10:07.643519 30832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 03:10:07.643546 30832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 03:10:07.643581 30832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 03:10:07.647490 30832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 03:10:07.647543 30832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 03:10:07.647562 30832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 03:10:07.647578 30832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 03:10:07.647591 30832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 03:10:07.653440 30832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.209.84:39575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.209.126:38311,127.7.209.125:42823,127.7.209.124:40855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/krb5kdc/kudu_127.7.209.124.keytab
--principal=kudu/127.7.209.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.209.124:40855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.209.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 29 Sep 2026 02:25:23 UTC on ccycloud.quasar-imoqlg.root.comops.site
build id 4679193
I20260929 03:10:07.654726 30832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 03:10:07.655922 30832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 03:10:07.664848 30841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:07.664857 30839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 03:10:07.664856 30838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 03:10:07.665076 30832 server_base.cc:1034] running on GCE node
I20260929 03:10:07.665834 30832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 03:10:07.666982 30832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 03:10:07.668133 30832 hybrid_clock.cc:648] HybridClock initialized: now 1790651407668117 us; error 29 us; skew 500 ppm
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 03:10:07.671211 30832 init.cc:377] Logged in from keytab as kudu/127.7.209.124@KRBTEST.COM (short username kudu)
I20260929 03:10:07.672103 30832 webserver.cc:460] Webserver started at http://127.7.209.124:45433/ using document root <none> and password file <none>
I20260929 03:10:07.672703 30832 fs_manager.cc:359] Metadata directory not provided
I20260929 03:10:07.672767 30832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 03:10:07.672962 30832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 03:10:07.675159 30832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/data/instance:
uuid: "a9006cccf8944abbbcad6b4df8d89281"
format_stamp: "Formatted at 2026-09-29 03:10:07 on dist-test-slave-bdps"
server_key: "7f04cbfe928777ebcfe23dba665faa4d"
server_key_iv: "e6a5155ca73025095d5004f5b06e52c0"
server_key_version: "encryptionkey@0"
I20260929 03:10:07.675719 30832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/wal/instance:
uuid: "a9006cccf8944abbbcad6b4df8d89281"
format_stamp: "Formatted at 2026-09-29 03:10:07 on dist-test-slave-bdps"
server_key: "7f04cbfe928777ebcfe23dba665faa4d"
server_key_iv: "e6a5155ca73025095d5004f5b06e52c0"
server_key_version: "encryptionkey@0"
I20260929 03:10:07.679601 30832 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 03:10:07.682080 30848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 03:10:07.683363 30832 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 03:10:07.683461 30832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/data,/tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/wal
uuid: "a9006cccf8944abbbcad6b4df8d89281"
format_stamp: "Formatted at 2026-09-29 03:10:07 on dist-test-slave-bdps"
server_key: "7f04cbfe928777ebcfe23dba665faa4d"
server_key_iv: "e6a5155ca73025095d5004f5b06e52c0"
server_key_version: "encryptionkey@0"
I20260929 03:10:07.683542 30832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 03:10:07.700999 30832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 03:10:07.704094 30832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 03:10:07.704278 30832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 03:10:07.711990 30832 rpc_server.cc:307] RPC server started. Bound to: 127.7.209.124:40855
I20260929 03:10:07.712258 30900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.209.124:40855 every 8 connection(s)
I20260929 03:10:07.713075 30832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/data/info.pb
I20260929 03:10:07.713408 8005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu as pid 30832
I20260929 03:10:07.713537 8005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskC1pyL7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790651120485886-8005-0/minicluster-data/master-2/wal/instance
I20260929 03:10:07.713946 8005 external_mini_cluster.cc:1428] Setting key 552ee1d4b8ad5dc1e5c817904c758067
I20260929 03:10:07.716681 30901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 03:10:07.720274 30751 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } attempt: 3
I20260929 03:10:07.721233 30901 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.124@KRBTEST.COM
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.126@KRBTEST.COM for kudu/127.7.209.124@KRBTEST.COM
I20260929 03:10:07.729365 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.720793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47302)
0929 03:10:07.721044 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.721061 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:07.721159 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:07.721414 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:07.721417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:07.721602 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:07.721759 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.721768 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.722636 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.722640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.723350 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.723362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.723910 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.723914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.723999 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.724492 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:07.724506 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:07.725964 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:07.727877 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.727881 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.727883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.728088 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.728346 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.728349 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.728351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.728438 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.728833 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:07.728836 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:07.728936 (+ 100us) client_negotiation.cc:769] Sending connection context
0929 03:10:07.729022 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 03:10:07.729177 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.124@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:10:07.730923 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.720843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47302 (local address 127.7.209.124:40855)
0929 03:10:07.721137 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.721146 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.721160 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:10:07.721253 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.721256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.721297 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.721375 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.721873 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.722524 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.723477 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.723777 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.724820 (+ 1043us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.726093 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.726096 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.726098 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.726119 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.727784 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.728185 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.728188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.728190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.728264 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.728517 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.728520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.728522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.728716 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.729566 (+ 850us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.729686 (+ 120us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.729794 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":131,"threads_started":1}
I20260929 03:10:07.730939 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.715522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47296 (local address 127.7.209.124:40855)
0929 03:10:07.715981 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.715998 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.716037 (+ 39us) server_negotiation.cc:407] Connection header received
0929 03:10:07.716786 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.716813 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.717217 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.717583 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.718499 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.719595 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.720604 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.721319 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.722012 (+ 693us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.724831 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.724838 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.724856 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.724888 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.726913 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.727414 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.727420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.727427 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.727553 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.727862 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.727866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.727867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.728445 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.728645 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.729125 (+ 480us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.729453 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":149,"threads_started":1}
I20260929 03:10:07.731666 30826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } attempt: 3
I20260929 03:10:07.735342 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.723647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.7.209.126:38311)
0929 03:10:07.723776 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.723780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.724226 (+ 446us) server_negotiation.cc:407] Connection header received
0929 03:10:07.724459 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.724463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.724516 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.724595 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.725560 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.726560 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.727793 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.728094 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.728187 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.731430 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.731434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.731436 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.731464 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.733303 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.733969 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.733972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.733973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.734056 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.734340 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.734343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.734345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.734642 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.734739 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.735070 (+ 331us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.735178 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":64}
I20260929 03:10:07.735358 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.723527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.126:38311 (local address 127.0.0.1:51744)
0929 03:10:07.724076 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.724133 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:07.724339 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:07.724787 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:07.724793 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:07.725261 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:07.725430 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.725438 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.726689 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.726693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.727670 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.727682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.728310 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.728314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.728426 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.728910 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:07.728924 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:07.731286 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:07.733521 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.733527 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.733532 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.733853 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.734161 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.734164 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.734166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.734249 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.734755 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:07.734762 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:07.734867 (+ 105us) client_negotiation.cc:769] Sending connection context
0929 03:10:07.734987 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 03:10:07.735185 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":87,"threads_started":1}
I20260929 03:10:07.736537 30901 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:10:07.737068 30751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207: Bootstrap starting.
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.126@KRBTEST.COM
I20260929 03:10:07.741200 30751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207: Neither blocks nor log segments found. Creating new log.
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.125@KRBTEST.COM for kudu/127.7.209.124@KRBTEST.COM
I20260929 03:10:07.742401 30751 log.cc:824] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207: Log is configured to *not* fsync() on all Append() calls
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.124@KRBTEST.COM for kudu/127.7.209.125@KRBTEST.COM
I20260929 03:10:07.744351 30907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.733788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51754 (local address 127.7.209.126:38311)
0929 03:10:07.734263 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.734276 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.734287 (+ 11us) server_negotiation.cc:407] Connection header received
0929 03:10:07.734317 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.734320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.734361 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.734456 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.735155 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.736136 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.737005 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.737321 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.737415 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.740278 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.740282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.740284 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.740308 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.741881 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.742387 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.742390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.742392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.742506 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.742828 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.742833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.742835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.743056 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.743942 (+ 886us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.744021 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.744142 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":415,"thread_start_us":83,"threads_started":1}
I20260929 03:10:07.745157 30751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207: No bootstrap required, opened a new log
I20260929 03:10:07.747193 30830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.732188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47308)
0929 03:10:07.732504 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.732519 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:07.732608 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:07.732842 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:07.732845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:07.733035 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:07.733167 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.733174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.738131 (+ 4957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.738135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.738990 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.739001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.739628 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.739631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.740231 (+ 600us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.740667 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:07.740681 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:07.742457 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:07.744776 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.744780 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.744782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.745035 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.745357 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.745360 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.745362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.745465 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.746541 (+ 1076us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:07.746545 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:07.746648 (+ 103us) client_negotiation.cc:769] Sending connection context
0929 03:10:07.746721 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 03:10:07.747013 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":493}
I20260929 03:10:07.747212 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.732465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47308 (local address 127.7.209.124:40855)
0929 03:10:07.732561 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.732565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.732578 (+ 13us) server_negotiation.cc:407] Connection header received
0929 03:10:07.732685 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.732687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.732726 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.732799 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.733313 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.737989 (+ 4676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.739141 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.739492 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.739771 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.742598 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.742601 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.742604 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.742627 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.744662 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.745136 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.745139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.745141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.745233 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.745945 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.745949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.745951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.746422 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.746863 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.746930 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.747046 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 03:10:07.749526 30751 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } }
I20260929 03:10:07.749950 30751 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6ea6ee0c8f04a9d80b5b4a194a0c207, State: Initialized, Role: FOLLOWER
I20260929 03:10:07.750377 30751 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } }
I20260929 03:10:07.751092 30910 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } } }
I20260929 03:10:07.751403 30910 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:07.751602 30826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f: Bootstrap starting.
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.209.125@KRBTEST.COM
I20260929 03:10:07.752230 30751 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:10:07.755347 30826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f: Neither blocks nor log segments found. Creating new log.
I20260929 03:10:07.756223 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.745667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34062 (local address 127.7.209.125:42823)
0929 03:10:07.746042 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.746052 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.746064 (+ 12us) server_negotiation.cc:407] Connection header received
0929 03:10:07.746092 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.746094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.746134 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.746971 (+ 837us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.747083 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.748491 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.749250 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.749533 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.749646 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.752302 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.752306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.752309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.752334 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.753910 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.754342 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.754346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.754348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.754428 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.754729 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.754733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.754734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.754955 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.755128 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.755870 (+ 742us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.755998 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":148,"threads_started":1}
I20260929 03:10:07.756433 30826 log.cc:824] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f: Log is configured to *not* fsync() on all Append() calls
I20260929 03:10:07.758282 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.737217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34052 (local address 127.7.209.125:42823)
0929 03:10:07.737350 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.737355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.737370 (+ 15us) server_negotiation.cc:407] Connection header received
0929 03:10:07.737412 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.737415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.737459 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.737526 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.737980 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.738674 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.739718 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.740061 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.740134 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.743225 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.743229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.743232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.743259 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.746874 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.748297 (+ 1423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.748301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.748303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.748391 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.757262 (+ 8871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.757266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.757268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.757481 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.757555 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.758006 (+ 451us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.758123 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260929 03:10:07.758759 30921 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:10:07.758828 30921 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:10:07.758987 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.737104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.125:42823 (local address 127.0.0.1:34052)
0929 03:10:07.737269 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.737282 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:07.737348 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:07.737547 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:07.737550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:07.737740 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:07.737847 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.737854 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.738822 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.738825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.739590 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.739601 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.740463 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.740467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.740580 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.741725 (+ 1145us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:07.741746 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:07.743074 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:07.747947 (+ 4873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.747950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.747952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.748183 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.756990 (+ 8807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.756994 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.756996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.757127 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.757684 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:07.757687 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:07.757810 (+ 123us) client_negotiation.cc:769] Sending connection context
0929 03:10:07.758621 (+ 811us) client_negotiation.cc:240] Negotiation successful
0929 03:10:07.758816 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260929 03:10:07.757802 30683 master_runner.cc:186] Error getting master registration for 127.7.209.125:42823: OK, instance_id { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" instance_seqno: 1790651407478831 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 03:10:07.761199 30683 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 03:10:07.761978 30901 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } has no permanent_uuid. Determining permanent_uuid...
I20260929 03:10:07.762799 8005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 03:10:07.764241 30826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f: No bootstrap required, opened a new log
I20260929 03:10:07.764191 30725 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51754
I20260929 03:10:07.764417 30725 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 03:10:07.764537 30725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:10:07.766069 30725 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } }
I20260929 03:10:07.768267 30725 leader_election.cc:288] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [CANDIDATE]: Term 1 election: Requested vote from peers 4db5252cf9b64eb39e6334f545d2d41f (127.7.209.125:42823), a9006cccf8944abbbcad6b4df8d89281 (127.7.209.124:40855)
I20260929 03:10:07.768668 30826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } }
I20260929 03:10:07.769133 30826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4db5252cf9b64eb39e6334f545d2d41f, State: Initialized, Role: FOLLOWER
Sep 29 03:10:07 dist-test-slave-bdps krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790651407, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.209.124@KRBTEST.COM for kudu/127.7.209.124@KRBTEST.COM
I20260929 03:10:07.769578 30826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } }
I20260929 03:10:07.771333 30826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:10:07.771421 30800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4db5252cf9b64eb39e6334f545d2d41f"
I20260929 03:10:07.772008 30800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:10:07.773640 30800 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6ea6ee0c8f04a9d80b5b4a194a0c207 in term 1.
I20260929 03:10:07.774199 30702 leader_election.cc:302] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4db5252cf9b64eb39e6334f545d2d41f, f6ea6ee0c8f04a9d80b5b4a194a0c207; no voters:
I20260929 03:10:07.774452 30906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.762508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.209.124:40855 (local address 127.0.0.1:47324)
0929 03:10:07.762612 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0929 03:10:07.762627 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 03:10:07.762716 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 03:10:07.764172 (+ 1456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 03:10:07.764175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 03:10:07.764379 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0929 03:10:07.764507 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.764515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.765484 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.765487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.766318 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 03:10:07.766329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.766991 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.766995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 03:10:07.767072 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.767841 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 03:10:07.767882 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0929 03:10:07.770287 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 03:10:07.772808 (+ 2521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.772811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.772813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.773044 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.773313 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 03:10:07.773316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 03:10:07.773318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 03:10:07.773401 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 03:10:07.773902 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 03:10:07.773905 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 03:10:07.773994 (+ 89us) client_negotiation.cc:769] Sending connection context
0929 03:10:07.774092 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 03:10:07.774279 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260929 03:10:07.774525 30910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 03:10:07.774752 30910 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [term 1 LEADER]: Becoming Leader. State: Replica: f6ea6ee0c8f04a9d80b5b4a194a0c207, State: Running, Role: LEADER
I20260929 03:10:07.774991 30905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.762639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.7.209.124:40855)
0929 03:10:07.763892 (+ 1253us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.763897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.763911 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:10:07.763944 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.763947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.763992 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.764068 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.764617 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.765363 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.766444 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.766741 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.766872 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.770500 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.770505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.770508 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.770530 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.772696 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.773142 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.773145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.773147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.773220 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.773495 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.773498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.773500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.773781 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.773872 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.774743 (+ 871us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.774857 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1174}
I20260929 03:10:07.775123 30910 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } }
I20260929 03:10:07.777256 30910 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6ea6ee0c8f04a9d80b5b4a194a0c207. Latest consensus state: current_term: 1 leader_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } } }
I20260929 03:10:07.777379 30910 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [sys.catalog]: This master's current role is: LEADER
I20260929 03:10:07.778365 30901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281: Bootstrap starting.
W20260929 03:10:07.781248 30939 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 03:10:07.781327 30939 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:10:07.781508 30901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281: Neither blocks nor log segments found. Creating new log.
I20260929 03:10:07.781646 30943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 03:10:07.782509 30901 log.cc:824] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281: Log is configured to *not* fsync() on all Append() calls
I20260929 03:10:07.784579 30927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } } }
I20260929 03:10:07.784727 30927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:07.782689 30758 master_runner.cc:186] Error getting master registration for 127.7.209.124:40855: OK, instance_id { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" instance_seqno: 1790651407709958 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 03:10:07.786337 30758 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 03:10:07.787406 30943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 03:10:07.791762 30901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281: No bootstrap required, opened a new log
I20260929 03:10:07.795511 30800 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [term 1 FOLLOWER]: Refusing update from remote peer f6ea6ee0c8f04a9d80b5b4a194a0c207: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:10:07.796368 30940 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:10:07.796454 30901 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } }
I20260929 03:10:07.796756 30901 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9006cccf8944abbbcad6b4df8d89281, State: Initialized, Role: FOLLOWER
I20260929 03:10:07.797166 30901 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } }
I20260929 03:10:07.798960 30901 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 03:10:07.799062 30875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a9006cccf8944abbbcad6b4df8d89281"
I20260929 03:10:07.799201 30874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [term 0 FOLLOWER]: Advancing to term 1
I20260929 03:10:07.800858 30874 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [term 1 FOLLOWER]: Refusing update from remote peer f6ea6ee0c8f04a9d80b5b4a194a0c207: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 03:10:07.801564 30940 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 03:10:07.802304 30875 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6ea6ee0c8f04a9d80b5b4a194a0c207 in term 1.
I20260929 03:10:07.804679 30927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6ea6ee0c8f04a9d80b5b4a194a0c207. Latest consensus state: current_term: 1 leader_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } } }
I20260929 03:10:07.804844 30927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:07.810710 30907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.771870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51764 (local address 127.7.209.126:38311)
0929 03:10:07.771989 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.771995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.785073 (+ 13078us) server_negotiation.cc:407] Connection header received
0929 03:10:07.785296 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.785300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.785355 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.785407 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.786511 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.787306 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.788569 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.788962 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.789072 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.793395 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.793399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.793402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.793430 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.796974 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.797862 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.797866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.797869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.798035 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.798527 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.798530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.798532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.805650 (+ 7118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.805806 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.810337 (+ 4531us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.810520 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 03:10:07.812139 30927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } } }
I20260929 03:10:07.812299 30927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:07.813903 30955 mvcc.cc:205] Tried to move back new op lower bound from 7334508166314119168 to 7334508166250848256. Current Snapshot: MvccSnapshot[applied={T|T < 7334508166314119168}]
I20260929 03:10:07.817418 30910 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } } }
I20260929 03:10:07.817565 30910 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [sys.catalog]: This master's current role is: LEADER
I20260929 03:10:07.818353 30943 catalog_manager.cc:1347] Generated new cluster ID: b98cfcc74c954e85be20fbd7e4be020a
I20260929 03:10:07.818419 30943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 03:10:07.818944 30948 mvcc.cc:205] Tried to move back new op lower bound from 7334508166314119168 to 7334508166250848256. Current Snapshot: MvccSnapshot[applied={T|T < 7334508166314119168 or (T in {7334508166314119168})}]
I20260929 03:10:07.823961 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.771177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47340 (local address 127.7.209.124:40855)
0929 03:10:07.771335 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.771340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.806863 (+ 35523us) server_negotiation.cc:407] Connection header received
0929 03:10:07.807068 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.807072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.807146 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.807203 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.808226 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.809089 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.815852 (+ 6763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.816262 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.816344 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.820343 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.820347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.820350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.820377 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.822100 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.822590 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.822594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.822596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.822680 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.822959 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.822962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.822964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.823196 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.823323 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.823652 (+ 329us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.823786 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 03:10:07.828740 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:07.783393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34072 (local address 127.7.209.125:42823)
0929 03:10:07.783509 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.783514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.809902 (+ 26388us) server_negotiation.cc:407] Connection header received
0929 03:10:07.810029 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.810032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.810092 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.810167 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.812172 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.813009 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.814231 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:07.814590 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:07.814665 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:07.825254 (+ 10589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 03:10:07.825258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 03:10:07.825260 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 03:10:07.825289 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 03:10:07.826917 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.827406 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.827410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.827412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.827495 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 03:10:07.827798 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 03:10:07.827802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 03:10:07.827803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 03:10:07.828017 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 03:10:07.828135 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:07.828474 (+ 339us) server_negotiation.cc:299] Negotiation successful
0929 03:10:07.828590 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 03:10:07.831494 30961 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } } }
I20260929 03:10:07.831633 30961 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:07.831717 30961 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6ea6ee0c8f04a9d80b5b4a194a0c207. Latest consensus state: current_term: 1 leader_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } } }
I20260929 03:10:07.831789 30961 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:07.835839 30910 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } } }
I20260929 03:10:07.835970 30910 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207 [sys.catalog]: This master's current role is: LEADER
I20260929 03:10:07.838534 30963 mvcc.cc:205] Tried to move back new op lower bound from 7334508166314119168 to 7334508166250848256. Current Snapshot: MvccSnapshot[applied={T|T < 7334508166314119168}]
I20260929 03:10:07.838724 30961 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6ea6ee0c8f04a9d80b5b4a194a0c207" member_type: VOTER last_known_addr { host: "127.7.209.126" port: 38311 } } peers { permanent_uuid: "4db5252cf9b64eb39e6334f545d2d41f" member_type: VOTER last_known_addr { host: "127.7.209.125" port: 42823 } } peers { permanent_uuid: "a9006cccf8944abbbcad6b4df8d89281" member_type: VOTER last_known_addr { host: "127.7.209.124" port: 40855 } } }
I20260929 03:10:07.838884 30961 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 03:10:07.850287 30966 catalog_manager.cc:1259] Loaded cluster ID: b98cfcc74c954e85be20fbd7e4be020a
I20260929 03:10:07.850389 30966 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281: loading cluster ID for follower catalog manager: success
W20260929 03:10:07.850967 30966 catalog_manager.cc:1576] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260929 03:10:07.851065 30966 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260929 03:10:07.856168 30943 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 03:10:07.857753 30943 catalog_manager.cc:1504] Loading token signing keys...
I20260929 03:10:07.868467 30943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6ea6ee0c8f04a9d80b5b4a194a0c207: Generated new TSK 0
I20260929 03:10:07.869530 30943 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 03:10:07.916923 30829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:07.913416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34088 (local address 127.7.209.125:42823)
0929 03:10:07.913597 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.913603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.915395 (+ 1792us) server_negotiation.cc:407] Connection header received
0929 03:10:07.915685 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.915690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.915770 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.915845 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.916707 (+ 862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34088: BlockingRecv error: recv got EOF from 127.0.0.1:34088 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260929 03:10:07.919358 30907 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:07.914574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51776 (local address 127.7.209.126:38311)
0929 03:10:07.914737 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.914742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.917803 (+ 3061us) server_negotiation.cc:407] Connection header received
0929 03:10:07.918406 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.918410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.918483 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.918546 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.919198 (+ 652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51776: BlockingRecv error: recv got EOF from 127.0.0.1:51776 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260929 03:10:07.921126 30904 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:07.918384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47354 (local address 127.7.209.124:40855)
0929 03:10:07.918525 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.918529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.920128 (+ 1599us) server_negotiation.cc:407] Connection header received
0929 03:10:07.920318 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.920322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.920382 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.920438 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.920988 (+ 550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47354: BlockingRecv error: recv got EOF from 127.0.0.1:47354 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260929 03:10:07.926682 30829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:07.924965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.7.209.125:42823)
0929 03:10:07.925086 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.925090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.925557 (+ 467us) server_negotiation.cc:407] Connection header received
0929 03:10:07.925883 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.925887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.925975 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.926028 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.926562 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34100: BlockingRecv error: recv got EOF from 127.0.0.1:34100 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260929 03:10:07.932541 30829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:07.930984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34102 (local address 127.7.209.125:42823)
0929 03:10:07.931093 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.931097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.931630 (+ 533us) server_negotiation.cc:407] Connection header received
0929 03:10:07.931830 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.931834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.931890 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.931939 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.932436 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34102: BlockingRecv error: recv got EOF from 127.0.0.1:34102 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260929 03:10:07.938282 30829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:07.936733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34104 (local address 127.7.209.125:42823)
0929 03:10:07.936831 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.936835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.937341 (+ 506us) server_negotiation.cc:407] Connection header received
0929 03:10:07.937529 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.937532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.937584 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.937669 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.938177 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34104: BlockingRecv error: recv got EOF from 127.0.0.1:34104 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260929 03:10:07.945679 30829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:07.944236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34116 (local address 127.7.209.125:42823)
0929 03:10:07.944355 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.944358 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.944871 (+ 513us) server_negotiation.cc:407] Connection header received
0929 03:10:07.945028 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.945031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.945077 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.945121 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.945558 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34116: BlockingRecv error: recv got EOF from 127.0.0.1:34116 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260929 03:10:07.956607 30829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:07.955623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34120 (local address 127.7.209.125:42823)
0929 03:10:07.955721 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.955725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.955925 (+ 200us) server_negotiation.cc:407] Connection header received
0929 03:10:07.956042 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.956044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.956086 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.956148 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.956520 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34120: BlockingRecv error: recv got EOF from 127.0.0.1:34120 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260929 03:10:07.976373 30829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:07.974776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34130 (local address 127.7.209.125:42823)
0929 03:10:07.974922 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:07.974927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:07.975200 (+ 273us) server_negotiation.cc:407] Connection header received
0929 03:10:07.975393 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:07.975398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:07.975462 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:07.975524 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:07.976236 (+ 712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34130: BlockingRecv error: recv got EOF from 127.0.0.1:34130 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260929 03:10:08.012084 30829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:08.010764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34144 (local address 127.7.209.125:42823)
0929 03:10:08.010940 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:08.010946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:08.011037 (+ 91us) server_negotiation.cc:407] Connection header received
0929 03:10:08.011209 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:08.011214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:08.011293 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:08.011423 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:08.011921 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34144: BlockingRecv error: recv got EOF from 127.0.0.1:34144 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260929 03:10:08.079833 30829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:08.078534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34154 (local address 127.7.209.125:42823)
0929 03:10:08.078743 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:08.078748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:08.078891 (+ 143us) server_negotiation.cc:407] Connection header received
0929 03:10:08.079036 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:08.079039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:08.079096 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:08.079182 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:08.079651 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34154: BlockingRecv error: recv got EOF from 127.0.0.1:34154 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260929 03:10:08.211450 30829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:08.210246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34158 (local address 127.7.209.125:42823)
0929 03:10:08.210437 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:08.210441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:08.210549 (+ 108us) server_negotiation.cc:407] Connection header received
0929 03:10:08.210697 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:08.210699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:08.210766 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:08.210847 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:08.211298 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34158: BlockingRecv error: recv got EOF from 127.0.0.1:34158 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260929 03:10:08.471457 30829 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 03:10:08.470041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34168 (local address 127.7.209.125:42823)
0929 03:10:08.470191 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:08.470195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:08.470322 (+ 127us) server_negotiation.cc:407] Connection header received
0929 03:10:08.470565 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:08.470569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:08.470635 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:08.470749 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 03:10:08.471313 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34168: BlockingRecv error: recv got EOF from 127.0.0.1:34168 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 03:10:08.783059 30939 catalog_manager.cc:1259] Loaded cluster ID: b98cfcc74c954e85be20fbd7e4be020a
I20260929 03:10:08.783181 30939 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f: loading cluster ID for follower catalog manager: success
I20260929 03:10:08.785117 30939 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f: acquiring CA information for follower catalog manager: success
I20260929 03:10:08.786525 30939 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4db5252cf9b64eb39e6334f545d2d41f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 03:10:08.854213 30966 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281: acquiring CA information for follower catalog manager: success
I20260929 03:10:08.855788 30966 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a9006cccf8944abbbcad6b4df8d89281: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 03:10:08.990403 31035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:08.985838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.7.209.125:42823)
0929 03:10:08.986139 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:08.986145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:08.986247 (+ 102us) server_negotiation.cc:407] Connection header received
0929 03:10:08.986419 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:08.986422 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:08.986602 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:08.986718 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:10:08.987032 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:08.987629 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:08.988619 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:08.988877 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:08.988974 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:08.989329 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:10:08.989949 (+ 620us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:10:08.990078 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:08.990137 (+ 59us) server_negotiation.cc:299] Negotiation successful
0929 03:10:08.990208 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":114,"threads_started":1}
I20260929 03:10:08.997175 31041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:08.993663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47362 (local address 127.7.209.124:40855)
0929 03:10:08.993926 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:08.993930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:08.993944 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:10:08.994046 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:08.994048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:08.994192 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:08.994306 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:10:08.994472 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:08.995001 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:08.995785 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:08.996005 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:08.996073 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:08.996303 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:10:08.996826 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:10:08.996920 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:08.996944 (+ 24us) server_negotiation.cc:299] Negotiation successful
0929 03:10:08.996994 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":94,"threads_started":1}
I20260929 03:10:09.004026 31041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:09.000380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47370 (local address 127.7.209.124:40855)
0929 03:10:09.000468 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:09.000471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:09.000681 (+ 210us) server_negotiation.cc:407] Connection header received
0929 03:10:09.000833 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:09.000837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:09.000881 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:09.000938 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:10:09.001214 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:09.001993 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:09.003060 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:09.003268 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:09.003353 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:09.003583 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:10:09.003670 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:10:09.003737 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:09.003827 (+ 90us) server_negotiation.cc:299] Negotiation successful
0929 03:10:09.003877 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260929 03:10:09.004245 31035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:09.000068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34182 (local address 127.7.209.125:42823)
0929 03:10:09.000191 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:09.000195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:09.000742 (+ 547us) server_negotiation.cc:407] Connection header received
0929 03:10:09.000831 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:09.000833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:09.000890 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:09.000953 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:10:09.001317 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:09.002141 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:09.002971 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:09.003312 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:09.003406 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:09.003713 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:10:09.003805 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:10:09.003869 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:09.004050 (+ 181us) server_negotiation.cc:299] Negotiation successful
0929 03:10:09.004115 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 03:10:09.004863 31048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 03:10:09.000081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51786 (local address 127.7.209.126:38311)
0929 03:10:09.000516 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0929 03:10:09.000520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 03:10:09.000534 (+ 14us) server_negotiation.cc:407] Connection header received
0929 03:10:09.000622 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 03:10:09.000625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 03:10:09.000819 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 03:10:09.000955 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 03:10:09.001141 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:09.001854 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:09.003012 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 03:10:09.003366 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 03:10:09.003453 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 03:10:09.003714 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 03:10:09.004278 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 03:10:09.004366 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 03:10:09.004630 (+ 264us) server_negotiation.cc:299] Negotiation successful
0929 03:10:09.004700 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":95,"threads_started":1}
I20260929 03:10:09.008013 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 30683
I20260929 03:10:09.015184 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 30758
I20260929 03:10:09.020798 8005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskC1pyL7/build/debug/bin/kudu with pid 30832
2026-09-29T03:10:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3055 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6136 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (288524 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST